-- Logs begin at Mon 2022-07-04 03:08:38 UTC, end at Mon 2022-07-04 04:38:04 UTC. -- Jul 04 04:00:37.181855 dv-cinder-742162-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 04:00:37.191406 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 04:00:37 2022] *** Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: nodename: dv-cinder-742162-5-iscsi Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: machine: x86_64 Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: clock source: unix Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: pcre jit disabled Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: detected number of CPU cores: 4 Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: current working directory: / Jul 04 04:00:37.193334 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: detected binary path: /usr/bin/uwsgi-core Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: your processes number limit is 47680 Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: your memory page size is 4096 bytes Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: detected max file descriptor number: 2048 Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: lock engine: pthread robust mutexes Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: thunder lock: enabled Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 04:00:37.193863 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 04:00:37.238311 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: Python main interpreter initialized at 0x56439f62b9c0 Jul 04 04:00:37.238311 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: python threads support enabled Jul 04 04:00:37.238311 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: your server socket listen backlog is limited to 100 connections Jul 04 04:00:37.238311 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: your mercy for graceful operations on workers is 90 seconds Jul 04 04:00:37.238745 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 04:00:37.238745 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: *** Operational MODE: preforking *** Jul 04 04:00:37.239055 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: *** uWSGI is running in multiple interpreter mode *** Jul 04 04:00:37.239055 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: spawned uWSGI master process (pid: 80360) Jul 04 04:00:37.239286 dv-cinder-742162-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 04:00:37.240985 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: spawned uWSGI worker 1 (pid: 80370, cores: 1) Jul 04 04:00:37.240985 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: spawned uWSGI worker 2 (pid: 80371, cores: 1) Jul 04 04:00:37.240985 dv-cinder-742162-5-iscsi devstack@keystone.service[80360]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 04:00:42.580018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 04:00:42.580018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: from cryptography.utils import int_from_bytes Jul 04 04:00:42.580908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 04:00:42.580908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: from cryptography.utils import int_from_bytes Jul 04 04:00:42.584730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 04:00:42.584730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: from cryptography.utils import int_from_bytes Jul 04 04:00:42.585238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 04:00:42.585238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: from cryptography.utils import int_from_bytes Jul 04 04:00:42.629485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 04:00:42.629670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 04:00:42.629883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] command line args: [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 04:00:42.630158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 04:00:42.630334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ================================================================================ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 04:00:42.630610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] admin_token = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.630862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_dir = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.631038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.631203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] config_source = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.631408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] debug = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.631574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.632934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.632934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 04:00:42.633125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 04:00:42.633260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] command line args: [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 04:00:42.633389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 04:00:42.633505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_config_append = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ================================================================================ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 04:00:42.633630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_dir = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] admin_token = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_dir = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.633985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] config_source = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] debug = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.634845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_config_append = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.635870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_dir = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.636912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] publish_errors = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.637932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.638966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] standard_threads = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_journal = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.639943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_json = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_stderr = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] use_syslog = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] publish_errors = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.640737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.640820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.641001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.641135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.641216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.641293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.641470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.641596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.641698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] standard_threads = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.641774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.641921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.642044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.642133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.642212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.642295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.642397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.642511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_journal = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.642685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_json = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.642807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.642886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_stderr = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.643038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.643123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] use_syslog = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.643224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.643398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.643668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.643800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 04:00:42.643892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.644897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.645862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.646899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.647866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.648935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.649951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.650895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.651980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652284 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.652981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.653828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.654909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.655942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.656864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.657983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.658996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.659238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.659548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.659634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.659792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.659868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.660984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.661933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662723 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.662953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.663858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.664996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.external = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.665929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.password = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] auth.token = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.666965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.667912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.668848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.external = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.669939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.password = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] auth.token = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.670939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.671989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.672937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.673890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.674034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.674112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.674330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.674428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.674761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.675927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.676972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.677983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.678961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.679941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.680955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.681900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.682876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.683951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.684974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.685901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.686872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.687961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.688969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.689970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.690199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.690575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.690657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.690828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.690919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.691874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.692960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.693990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.694952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.695890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.696979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.697915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.698944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.699894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.700928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.701941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.702930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.703904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.704887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.705889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.706901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.707858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.708918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.709963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.710937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.711916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.712865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.713946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.714913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.715921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.716071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.716148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.716228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 04:00:42.716322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.716554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.716798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.717097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.717359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.717606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.717853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.718093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.718335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.718598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.718837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.719075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.719312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.719559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.719796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.720041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.720276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.720523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.720760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.721005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.721393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.722075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.722314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 04:00:42.722525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 04:00:42.746741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.746969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.747208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.748852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.749091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.749323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.755492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.758755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.783696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.783976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.785898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.786132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.926519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.929409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.974128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.974427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.980304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:42.980546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.038489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.038737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.038906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.039860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.040080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.040305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.063533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.063682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.063774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.063979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.064831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.065222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.075779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.076579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.088460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.088708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.088883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.089964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.090218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.090434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.090686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 04:00:43.106985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.107117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.107187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 04:00:43.107249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 04:00:43.107501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.107629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.107699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 04:00:43.107782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 04:00:43.107946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.108920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.119667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.119942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.120198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.120475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.120620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.120741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.121028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.121309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.128727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.130028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.132003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.132277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.132548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.132821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.133619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.133943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.134112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.134206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.134743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.134869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.134977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.135089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.135384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.135652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.135920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.136176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.136441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.161294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.162918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.163202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.163648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.165268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.165545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.168427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.168701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.169088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.170842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.171125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.171404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.176401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.176730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.179420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.179705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.181136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.181418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.181696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.181972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.182239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.182530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.182762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.183045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.184188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.184479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.184754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.185031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.185305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.185563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.185836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.186101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.201681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.201955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.205276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.205737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.206667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.210260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.210462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.210590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.214129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.214402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.215630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.219315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.219611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.223166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.223453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.223579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.223738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.224019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.224296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.224569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.227208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.227507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.227788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.228069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.228353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.228621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.243695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.243982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.244254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.244508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.248285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.248564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.248832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.249111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.252332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.254055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.254327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.257334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.257476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.257744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.258063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.258334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.258647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.259085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.259673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.262829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.263110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.263511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.263794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.264075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.272141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.272414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.272691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.272963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.273239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.274790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.275082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.277533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.277815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.278095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.278373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.278673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.278945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.279227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.292935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.296202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.299605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.300080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.301354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.302997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.303272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.303414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.303552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.306740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.308347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.310116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.310392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.310699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.310823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.312421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.315882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.316170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.316553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.316832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.318719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.320341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.323591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.323884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.324165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.324460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.325865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.326147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.326416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.334176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.334473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.334600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.334791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.334915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.335066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.335284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.335548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.335921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.336191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.336462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.336743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.337002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.337258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.337523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.337785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.338053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.342630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 04:00:43.342911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.343186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.343458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.343726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.343990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.344267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.344551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.344822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.345187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.345463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.345724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.345998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.346765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 04:00:43.375519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56439f62b9c0 pid: 80370 (default app) Jul 04 04:00:43.378991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c7594-c1dd-47cd-9ff7-8c1d27b2940e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:43.379279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c7594-c1dd-47cd-9ff7-8c1d27b2940e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:43.379570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c7594-c1dd-47cd-9ff7-8c1d27b2940e None None] PATH_INFO: `/v3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:43.383738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Mon Jul 4 04:00:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:43.385644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56439f62b9c0 pid: 80371 (default app) Jul 04 04:00:51.040824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:51.041403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:51.041403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:51.043464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:51.053597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:51.054069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:51.054159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:51.121649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: INFO dbcounter [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] Registered counter for database keystone Jul 04 04:00:51.145458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG oslo_db.sqlalchemy.engines [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 04:00:51.156321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writer thread running {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 04:00:51.345237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG passlib.handlers.bcrypt [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 04:00:51.345548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG passlib.handlers.bcrypt [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 04:00:51.407277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:51.409981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:51.543082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 04:00:51.748788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:51.825442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: INFO dbcounter [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] Registered counter for database keystone Jul 04 04:00:51.847261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG oslo_db.sqlalchemy.engines [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 04:00:51.858793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writer thread running {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 04:00:51.982258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37e2a7-5699-4c38-b761-6a9127ff92ae None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:51.988161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:51.988827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:51.989109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:51.989389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:51.998522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:52.050331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING keystone.common.rbac_enforcer.enforcer [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 04:00:52.050967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e26e9-7681-4ab5-9f86-16c091af18d3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:52.061160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Jul 4 04:00:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:00:52.070954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:52.089411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9c28e-1ff4-4bf1-803a-8fd2cf187e86 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:52.095602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:52.097017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:52.097308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:52.097589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] PATH_INFO: `/v3/projects/525d4bbd65cc40f382674c9a7977e83e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:52.101935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] RBAC: Authorizing `identity:get_project(project_id=525d4bbd65cc40f382674c9a7977e83e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:52.152306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 04:00:52.152655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7561e07-f56b-4e4b-bac7-3a265d7e423d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:52.157298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 2/5] 10.222.0.16 () {56 vars in 1207 bytes} [Mon Jul 4 04:00:52 2022] GET /identity/v3/projects/525d4bbd65cc40f382674c9a7977e83e => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:54.150640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672cb6-9956-4bc8-8f72-8d5498a47a3d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:54.151212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672cb6-9956-4bc8-8f72-8d5498a47a3d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:54.151212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672cb6-9956-4bc8-8f72-8d5498a47a3d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:54.155294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 4/6] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:54.162160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:54.162933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:54.163074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:54.188934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:54.190691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:54.200447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 04:00:54.223946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de672cb6-9956-4bc8-8f72-8d5498a47a3d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:54.243001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de672cb6-9956-4bc8-8f72-8d5498a47a3d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:54.248390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:54.249021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:54.249301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:54.249575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:54.258551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:54.260652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bb426-f49a-469d-b52f-a6190ff6e156 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:54.270594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1324 bytes} [Mon Jul 4 04:00:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:00:54.276382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:54.286014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e256-eb9c-4c77-a07e-f9fea2d27394 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:54.291188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:54.292065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:54.292347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:54.292620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:54.294149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:54.296022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b27439-b0a3-4e26-b10d-32bc8201dc8c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:54.297593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 4/9] 10.222.0.16 () {56 vars in 1198 bytes} [Mon Jul 4 04:00:54 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:56.356152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056eb3-6feb-49e7-a17f-723e6b33a40f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.356750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056eb3-6feb-49e7-a17f-723e6b33a40f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.356856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056eb3-6feb-49e7-a17f-723e6b33a40f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.358254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:56.363948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.364388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.364479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.382253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:56.383560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:56.408511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb588068-278e-4069-ac6f-a6d193923700 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.408871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb588068-278e-4069-ac6f-a6d193923700 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.409357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb588068-278e-4069-ac6f-a6d193923700 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.411617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 7/11] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:56.419423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.419833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.420217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.460905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG passlib.handlers.bcrypt [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80370) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 04:00:56.461321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG passlib.handlers.bcrypt [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80370) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 04:00:56.478537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 5/12] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:00:56.501870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:56.507648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:56.509233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:56.526636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:56.607088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc943d39-4f4c-4571-aa35-b85d5a64dcb2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:56.610598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 8/13] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:00:56.636517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:56.657252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:56.658064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.658347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.658879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.670297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:00:56.681715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:00:56.682663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:00:56.685951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 6/14] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:00:56.688845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:56.737513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69933-83c0-40fa-920b-cf3c42ab8cdc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:56.739831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:56.770445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:56.775924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:56.776839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.777125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.777402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.790322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ddc-8f4c-4dca-b310-3059f08eeef9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:56.824154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:56.824827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.825112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.825375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.826151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:00:56.826489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:56.827896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:56.840306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 7/15] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:00:56.844430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:56.845961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:56.846102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:56.947285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be2fb4-6ba1-4ef2-a0c3-2a7dc4b8415f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:56.949262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 9/16] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 04:00:56.955861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:56.958951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:56.982313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:56.983238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:56.983523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:56.983796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:56.984998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:56.986534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:56.988298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 8/17] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:00:56.993857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:57.008390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:57.054526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf2351-ee63-4058-9ff5-f8e28ea50a74 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:57.089265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:57.090060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:57.090341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:57.090641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:57.093367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cfbe-44cf-458c-a7a9-4fab4c322746 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:57.110201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:57.111871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:57.128543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:57.129344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:57.129632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:57.129913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:57.131138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:57.132577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:00:57.132795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:00:57.133570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:00:57.134341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:00:57.134341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:00:57.134341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:00:57.134341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:00:57.134341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:00:57.134950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:00:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:57.140551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:57.140679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:57.198926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:57.220669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:57.239629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bde32-ec4f-4a42-80e3-1b9bf31248d3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:57.245100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5a64-ff97-491b-b1cf-5c3f3141a993 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:57.273708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:57.274402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:57.274685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:57.274955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:57.275798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.domain.id=default, user.id=769a75081be74879911bd8c4f4cb57da) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:00:57.276111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:57.277553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb9f03-d5e2-4cab-9b3d-7352ba839e5f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:57.279514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:57.280175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:57.280456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:57.280730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:57.281481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:00:57.281791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:00:57.283179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f5e5-ebdf-4322-a69a-c3bbad8e0141 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:00:57.284703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1322 bytes} [Mon Jul 4 04:00:57 2022] GET /identity/v3/role_assignments?user.id=769a75081be74879911bd8c4f4cb57da&scope.domain.id=default&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:57.303363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:00:59.614339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d041b6-7bcc-463d-8ef7-6b74abbc0172 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:59.614880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d041b6-7bcc-463d-8ef7-6b74abbc0172 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:59.614984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d041b6-7bcc-463d-8ef7-6b74abbc0172 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:59.616304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:59.621814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec456-3155-449a-b568-47360a3d2bbd None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:59.622679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec456-3155-449a-b568-47360a3d2bbd None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:59.623187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec456-3155-449a-b568-47360a3d2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:59.651405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c49ec456-3155-449a-b568-47360a3d2bbd None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:59.652966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c49ec456-3155-449a-b568-47360a3d2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:59.741207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:00:59.784202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d041b6-7bcc-463d-8ef7-6b74abbc0172 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:00:59.821233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000387f-1b76-4845-8323-230d7b57f8a4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:59.821888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000387f-1b76-4845-8323-230d7b57f8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:59.822415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000387f-1b76-4845-8323-230d7b57f8a4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:59.824010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:00:59.829021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1838-6222-483b-9600-46860458d107 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:59.829544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1838-6222-483b-9600-46860458d107 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:59.830037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1838-6222-483b-9600-46860458d107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:00:59.879157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-092b1838-6222-483b-9600-46860458d107 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:00:59.880694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-092b1838-6222-483b-9600-46860458d107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:00:59.948008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d041b6-7bcc-463d-8ef7-6b74abbc0172 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:00:59.987408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:00:59.994933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:00:59.995771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:00:59.996058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:00:59.996339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.020804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-5b7d78e3-6848-4b4b-b6b3-1ec6f3a92642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:00.028846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:00.029774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:00.030548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 12/25] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:00:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:00.032648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a57e5-e33a-4080-92d9-63bc9d83507f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.033154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a57e5-e33a-4080-92d9-63bc9d83507f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.036621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a57e5-e33a-4080-92d9-63bc9d83507f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.041530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 14/26] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 04 04:01:00.046689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1838-6222-483b-9600-46860458d107 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.051916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 13/27] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:00 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:00.059930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.060471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.060934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.083065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:00.103526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:00.105258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:00.216124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:00.217691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.250075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1838-6222-483b-9600-46860458d107 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.315577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.316726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.317955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.318623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.320738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:00.321997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.324239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.346298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 15/29] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:00.352101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.395227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e806506-dbb1-4ef4-9460-a44cd81aad9a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.426510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:00.449913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.451269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.451568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.451901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.454324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:00.467091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.468932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:01:00.493524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:00.494415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:00.495350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:01:00.495350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:01:00.495350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:01:00.495350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:00.495350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:00.495594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.498634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad897200-93d4-42e6-97a3-1b0d00e8f70d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.535871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.536918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.537422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.537888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.538898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.540556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.614758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea1942-422f-4142-b3bd-7658e2e6c1b2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.656537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479819c-c648-458d-a67c-189a410637ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.657972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479819c-c648-458d-a67c-189a410637ec None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.658549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479819c-c648-458d-a67c-189a410637ec None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.659075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479819c-c648-458d-a67c-189a410637ec None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.660618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479819c-c648-458d-a67c-189a410637ec None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.662401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479819c-c648-458d-a67c-189a410637ec None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.664306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:00 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:00.665792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286656e-402d-4101-a667-91597a41d657 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.666273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286656e-402d-4101-a667-91597a41d657 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.666832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286656e-402d-4101-a667-91597a41d657 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.668330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 17/32] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:00.673578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286656e-402d-4101-a667-91597a41d657 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.700350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 16/33] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Jul 4 04:01:00 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:00.701360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:00.785009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:00.856870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:00.864046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e83d6-481b-438f-9b04-209e92ab104a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.882788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286656e-402d-4101-a667-91597a41d657 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:00.909079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.910054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.910485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.910863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.912724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.914804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d607b9-b0f3-4a95-a7a4-b4bf7da38cf4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.921497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 17/34] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:00.923206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.923591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.923745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:00.924188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.924997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:00.925421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:00.925844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:00.926807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:00.927274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:00.928985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:00.943775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:00.945492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:00.948461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 18/35] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:00.949505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.043341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 18/36] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:01.043964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.073250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bd2f3-7022-4d51-abaa-8ef656d29b0a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.089062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:01.115312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.116215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.116500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.116781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] PATH_INFO: `/v3/domains/default/users/769a75081be74879911bd8c4f4cb57da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.118896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=769a75081be74879911bd8c4f4cb57da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.120371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55e41-e8c4-449b-86d5-25850c5a54db None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.152475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:01.159419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e0523-ec12-499d-bd90-a2489c567f97 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.195315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.195979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.196262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.196537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.197249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.198718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.225570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 19/37] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/projects => generated 301 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:01.226346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.234773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 19/38] 10.222.0.16 () {58 vars in 1361 bytes} [Mon Jul 4 04:01:00 2022] PUT /identity/v3/domains/default/users/769a75081be74879911bd8c4f4cb57da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:01.236371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3e8c-e560-405a-bd04-187a43402882 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.236734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3e8c-e560-405a-bd04-187a43402882 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.237531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3e8c-e560-405a-bd04-187a43402882 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.246007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 20/39] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:01.247556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040568-b98d-47f9-b910-9971b903b2d7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.247851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040568-b98d-47f9-b910-9971b903b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.248136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040568-b98d-47f9-b910-9971b903b2d7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.254471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 21/40] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:01.254471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689573b-3b34-46b0-b254-094357d6a635 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.254471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689573b-3b34-46b0-b254-094357d6a635 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.254471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689573b-3b34-46b0-b254-094357d6a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.290999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9689573b-3b34-46b0-b254-094357d6a635 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:01.293898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9689573b-3b34-46b0-b254-094357d6a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:01.371552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeda74-65f7-4849-82ef-8156b15bdca9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.396022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 22/41] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:01.397784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.398297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.398808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.421387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:01.424052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:01.428510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.429261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.430450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.430450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.431629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.433049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.449182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 20/42] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/roles => generated 219 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:01.450014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.464071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:01.581334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b3ac1-9e00-4080-a2f9-cb4c05d73467 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.624958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.626056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.626541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.626983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.631260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.633731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.637203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 21/43] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:01.643971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.644992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 23/44] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:01.667455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:01.704161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:01.746398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:01.774852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a1fd-c69e-4f5f-a98e-79860a872fff None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.780546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47705d8f-6cbf-4442-b680-8756a36229eb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:01.812439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.813109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.813394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.813677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.814467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.815901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337e3c-5756-43e5-b677-58c3cb269e91 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.816378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:01.817109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:01.817368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:01.817646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:01.818444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:01.819985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e15309-240d-42c4-a4cf-799b2d76b197 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:01.840259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 24/45] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/roles => generated 223 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:01:01.846096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 22/46] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:01:01 2022] POST /identity/v3/projects => generated 312 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:01:03.369920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb9292-8e4d-4c06-a396-37cc66f3657d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.370508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb9292-8e4d-4c06-a396-37cc66f3657d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.370508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb9292-8e4d-4c06-a396-37cc66f3657d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.371906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 25/47] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:03.377135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.377571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.377916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.395764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:03.396971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:03.467349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 23/48] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:03.491231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb9292-8e4d-4c06-a396-37cc66f3657d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:03.592009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:03.600849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb9292-8e4d-4c06-a396-37cc66f3657d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:03.639022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:03.639838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.640129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.640412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.651093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:03.659279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:03.660131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:03.660968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 26/49] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:03.666436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:03.726721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:03.772414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec906f-f7da-42e7-8e31-7ae6abd7e478 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:03.807040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:03.807711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.807992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.808274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.809035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:03.809347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:03.810771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb0048-4120-401f-be00-19d1aeb717db None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:03.819026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 24/50] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:03.824683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:03.907702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddfba-1188-4b37-a2ff-ceae489f538e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.907902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddfba-1188-4b37-a2ff-ceae489f538e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.908168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddfba-1188-4b37-a2ff-ceae489f538e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.909853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 25/51] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:03.914499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.914802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.915088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.927067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f1d99-697c-4b60-ac72-d98e5d65d3d5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:03.933359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:03.934609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:03.962465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:03.963366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:03.963655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:03.963939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:03.965177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:03.966677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:03.968417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 27/52] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:03 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:03.973891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:03.999992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:04.086622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de228-1b38-4352-8975-0c2f3143ef2c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:04.110343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 26/53] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:04.120038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:04.120846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:04.121134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:04.121414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:04.122639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:04.124101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:04.125742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:04.131513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:04.134278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:04.181723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:04.231631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:04.231829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b7115-ff90-445a-aeb3-3c5c431e6aa0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:04.242517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfba23-85d3-422b-9cab-a28e4ed4c1ee None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:04.266961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:04.267685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:04.267997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:04.268263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:04.269118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.domain.id=default, user.id=769a75081be74879911bd8c4f4cb57da) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:04.269424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:04.270912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:04.276828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:04.277632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:04.277923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:04.278203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:04.279440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1322 bytes} [Mon Jul 4 04:01:04 2022] GET /identity/v3/role_assignments?user.id=769a75081be74879911bd8c4f4cb57da&scope.domain.id=default&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:04.279534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:04.280968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26590e4b-38e9-4db4-a3d4-fe43c5ae2515 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:04.282617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:04.288613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:04.399764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536fc7b-341d-4225-8fb5-1396229c2fd7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:04.415674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:04.439519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:04.440253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:04.440540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:04.440817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:04.441540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:04.443077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b5789-b548-4229-b3dc-bf6184a0b6bb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:04.469638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Jul 4 04:01:04 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:06.455041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc2de-fc97-4afe-94ed-a88d16591540 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:06.455591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc2de-fc97-4afe-94ed-a88d16591540 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:06.456051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc2de-fc97-4afe-94ed-a88d16591540 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:06.457195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:06.464491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:06.464900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:06.465061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:06.484836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:06.486059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:06.562584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:06.594129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bc2de-fc97-4afe-94ed-a88d16591540 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:06.708216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bc2de-fc97-4afe-94ed-a88d16591540 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:06.743690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:06.744490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:06.744778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:06.745054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:06.746352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:06.747831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fed0bd-3aab-48a3-9d1a-8ef9e5eea64e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:06.749485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:06.755995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:06.859519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:06.868380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a0ac9-8e86-45c0-8d1e-a966ef34282c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:06.915988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:06.916663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:06.916948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:06.917232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:06.917946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:06.919428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c968bdc-5009-44b4-abce-107a49c530a6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:06.950279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:06 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:09.984806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90dabe-4ef4-46b6-8f4e-11d00b379efc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:09.984806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90dabe-4ef4-46b6-8f4e-11d00b379efc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:09.984806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90dabe-4ef4-46b6-8f4e-11d00b379efc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:09.985858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:09.991512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd74c7-21f9-46d9-b059-b72b6728d2b3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:09.992311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd74c7-21f9-46d9-b059-b72b6728d2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:09.992803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd74c7-21f9-46d9-b059-b72b6728d2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.014124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-27dd74c7-21f9-46d9-b059-b72b6728d2b3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:10.015382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-27dd74c7-21f9-46d9-b059-b72b6728d2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:10.112748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:10.149045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90dabe-4ef4-46b6-8f4e-11d00b379efc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.167332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d48cc-5894-4e1a-9fc5-8f93eb04adf9 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.167953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d48cc-5894-4e1a-9fc5-8f93eb04adf9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.168438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d48cc-5894-4e1a-9fc5-8f93eb04adf9 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.170091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 32/64] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:10.175986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750a2d-cabe-4ae2-be0f-64ac7700878d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.176543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750a2d-cabe-4ae2-be0f-64ac7700878d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.177024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750a2d-cabe-4ae2-be0f-64ac7700878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.184411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.209616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7b750a2d-cabe-4ae2-be0f-64ac7700878d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:10.210839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7b750a2d-cabe-4ae2-be0f-64ac7700878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:10.296388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90dabe-4ef4-46b6-8f4e-11d00b379efc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.315538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 33/65] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:10.317475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee10d-d09a-4378-8bbc-1443b68b6d39 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.317900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee10d-d09a-4378-8bbc-1443b68b6d39 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.318314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee10d-d09a-4378-8bbc-1443b68b6d39 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.320297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 34/66] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:10.326131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.326629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.327445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.337841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.338940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.339386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.339797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.347255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:10.349120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:10.352515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:10.357421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:10.361828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:10.362965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:10.363914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:10.365948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 33/67] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:10.367302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.437303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 35/68] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:10.438363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.483323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.500771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3504-f5b6-41dd-9d7e-bdd0cf8ced09 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.540611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.541749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.542171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.542755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.553932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:10.562981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:10.563872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:10.564730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:10.565291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda2b5-37f7-4807-b6ea-88b80ea2dd2a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.567712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:10.568712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.613855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.615197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.615576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.615926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.616772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:10.617150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:10.619098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:10.635511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 36/70] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:10.639197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.645145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.709895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4766f0-3a53-422d-ba94-9ed7113acd09 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.748362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.749223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.749500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.749775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.752237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940e2f-74e1-43c7-8d88-5a8dcd6879db None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.760797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:10.769342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:10.770189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:10.775946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.784703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.793387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.794081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.794411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.794682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.795457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:10.795774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:10.797180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745e84b-e867-4ebb-9b17-57297256f190 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:10.807310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 37/72] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:10.808409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cd29-a8be-4369-80f5-1ac93aa06677 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.808705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cd29-a8be-4369-80f5-1ac93aa06677 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.808989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cd29-a8be-4369-80f5-1ac93aa06677 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.810406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 38/73] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:10.815167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cd29-a8be-4369-80f5-1ac93aa06677 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.886691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:10.888399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3308c95-0715-472f-a596-6925fc992073 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.921231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.922127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.922445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.922728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.922838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cd29-a8be-4369-80f5-1ac93aa06677 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:10.936531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:10.938140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:10.940441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 36/74] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:10.941215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:10.962339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:10.963042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.963351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.963628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.964384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:10.964695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:10.966076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecda24-703f-43ac-9cf5-4cb99bb54c42 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:10.974520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 39/75] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:10.975694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d08-de5a-4b40-bc5d-0474e65472c6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:10.975974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d08-de5a-4b40-bc5d-0474e65472c6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:10.976243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d08-de5a-4b40-bc5d-0474e65472c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:10.992325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9b9a3d08-de5a-4b40-bc5d-0474e65472c6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:10.993526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9b9a3d08-de5a-4b40-bc5d-0474e65472c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:11.005709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.043863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb08c3-75e0-4561-a823-417847123004 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.061596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 40/76] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:11.063177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b4df-c2f4-49d9-8a4a-65f3389a98fc None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.063536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b4df-c2f4-49d9-8a4a-65f3389a98fc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.063805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b4df-c2f4-49d9-8a4a-65f3389a98fc None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.065164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 41/77] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:11.065840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94b4df-c2f4-49d9-8a4a-65f3389a98fc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.090541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.091663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.091967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.092242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.093476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.094971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.096647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 37/78] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:10 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:11.097407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.134942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.176950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94b4df-c2f4-49d9-8a4a-65f3389a98fc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.194833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70857b34-edcf-4759-baa2-f514df0cc266 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.210546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.211204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.211490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.211766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.223302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.228998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.229880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.230160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.230458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.231680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.233128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c2dbb-9a82-4827-8cf7-99c4d4cf4517 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.234807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 38/79] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:11.235993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.236364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.236651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.254276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:11.255500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:11.279691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.281147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.324106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 39/80] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:11.324853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.338447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.381165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 42/81] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:11.382004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.422077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5ade6-fc26-4fb5-9bff-82bfb9a8775f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.449432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.456515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.457332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.457618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.457902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.478585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:11.481285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4ac0e-e39c-49d2-bdd7-2433a7f4cde3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:11.486267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:11.487467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 40/82] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:11.493045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.522123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.522809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.523099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.523378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.552856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.593383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.594824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.595634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d6f47-949f-4813-ad4e-779b676870a3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.631635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.632305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.632590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.632865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.691748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.692495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 43/83] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:11.698265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.702921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.704405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.754258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.795566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b2f94-f415-44f7-92ac-7d16b50a9ed4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.805852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 41/84] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:11.806898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.838049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.838885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.839166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.839441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.840627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.842092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.843796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 44/85] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:11.844497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.907498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.917432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e21e9-28a2-4a47-a2dc-b85100ad5ebe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.948635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c326d1-7c71-4ffc-a74b-6529dccaad70 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:11.951501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.952389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.952678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.952958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.954190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.956020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:11.957709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 42/86] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:11.958518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:11.979466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:11.988962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:11.989671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:11.989957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:11.990238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:11.991107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:11.991420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:11.992796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.001113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 45/87] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.001892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.061270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe546-d27e-44e2-a112-8f6b825464e6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.094477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.095286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.095567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.095866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.097061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.098545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.100086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbc62-6a72-433b-8e13-99be346c29f5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.100231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 43/88] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:11 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.100968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.132538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.133691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.133982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.134256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.135834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.137276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.138964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 46/89] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.139718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.146910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:12.198098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:12.220460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef509240-7afb-41db-9456-2ecd327454f1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.242068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a5996-1af0-4f29-8dc3-0efd1fdc073c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.253646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.254330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.254671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.254949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.255680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.257162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.274572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.275360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.275706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.276057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.276926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:12.277275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.278876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.283942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 44/90] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:12 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:12.293858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=d4760ffd4761422abfabd45bc0880415 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.294116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.294195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.393074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:12.415605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3939ea-a0a9-4296-bdc9-3fc953b3edbf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.432112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfbca-3e28-4af0-aafb-80e4faf58cd4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.457656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.458220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:12.458648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.458934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.459218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.460940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.462157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.464038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 45/92] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.464851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.476819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.477491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.477776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.478055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.478946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=769a75081be74879911bd8c4f4cb57da) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:12.479264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.480734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.487907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 48/93] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/role_assignments?user.id=769a75081be74879911bd8c4f4cb57da&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.488799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.628095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c90433-45e4-452f-9157-8c492acab040 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.635426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598930a-9f1e-4be1-91da-7d8ac587648f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:12.682868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.683489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.683781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.684061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.684914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:12.685171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:12.685279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.685836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:12.686124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:12.686433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:12.686773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.694349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:12.754932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:12.775165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:12.776889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:12.890241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 49/95] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:12.896184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:12.915028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:13.019863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae01eb-ef16-4c89-8026-ba46f7f05ce2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:13.084030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:13.084924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:13.085211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:13.085490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] PATH_INFO: `/v3/projects/a7de582a100c418393f9fae9b17d0f35` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:13.091133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] RBAC: Authorizing `identity:get_project(project_id=a7de582a100c418393f9fae9b17d0f35)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:13.092630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4b3c9-1888-4275-93e0-5448fb8ea99d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:13.094406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:12 2022] GET /identity/v3/projects/a7de582a100c418393f9fae9b17d0f35 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:13.100548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:13.239997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:13.264537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746adf7-117d-44ae-998b-26aae90b1ffd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:13.310198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:13.310810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:13.311096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:13.311375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:13.312219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=a7de582a100c418393f9fae9b17d0f35, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:13.312533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:13.313968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfa88b-b02a-40b1-9d90-6c5a9f854c85 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:13.322838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:13 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=a7de582a100c418393f9fae9b17d0f35&role.id=d4760ffd4761422abfabd45bc0880415 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:15.422563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0021a3-d165-4cd6-bde9-616d559edf38 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.423080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0021a3-d165-4cd6-bde9-616d559edf38 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.423153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0021a3-d165-4cd6-bde9-616d559edf38 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.424535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:15.430183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f87448-3308-418a-8ab7-4b81ffe32a8d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.430664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f87448-3308-418a-8ab7-4b81ffe32a8d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.430836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f87448-3308-418a-8ab7-4b81ffe32a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.441617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b12d-c9d2-4d74-bd9a-d7d58e9c12b9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.442448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b12d-c9d2-4d74-bd9a-d7d58e9c12b9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.442744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b12d-c9d2-4d74-bd9a-d7d58e9c12b9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.445912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 49/99] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:15.450238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7f1b9-cdba-476b-bf6b-510a8c8f1c4e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.450555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7f1b9-cdba-476b-bf6b-510a8c8f1c4e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.450844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7f1b9-cdba-476b-bf6b-510a8c8f1c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.476223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a0f87448-3308-418a-8ab7-4b81ffe32a8d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:15.477475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a0f87448-3308-418a-8ab7-4b81ffe32a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:15.488166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-77e7f1b9-cdba-476b-bf6b-510a8c8f1c4e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:15.492022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-77e7f1b9-cdba-476b-bf6b-510a8c8f1c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:15.605927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 51/100] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:15.607853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc918-33c9-4b51-9555-64c62b32fdcd None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.608147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc918-33c9-4b51-9555-64c62b32fdcd None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.608430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc918-33c9-4b51-9555-64c62b32fdcd None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.609839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 52/101] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:15.610812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d37d-0f81-473f-9c54-d074db75f3b5 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.611100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d37d-0f81-473f-9c54-d074db75f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.611259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 50/102] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:15.611432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d37d-0f81-473f-9c54-d074db75f3b5 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.612851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 53/103] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:15.615974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.616480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.617172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.617994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.618287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.618619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.640334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:15.641871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:15.645027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:15.646455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:15.655631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:15.723806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 51/104] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:15.725121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:15.726772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 54/105] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:15.727487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:15.761996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:15.860469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcee03-a7d1-4af7-86aa-a4d0b20e9133 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:15.888834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3331-9dff-4ee5-993e-bb32e4766083 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:15.898637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:15.899811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.900087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.900676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.912648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:15.921710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:15.922589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:15.923353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 55/106] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:15.924098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:15.926566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:15.927389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:15.927682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:15.927964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:15.939020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:15.947515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:15.948620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:15.949397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 52/107] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:15.950634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:15.952758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.006463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.044295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e10de6-883e-4455-b940-b8ac1111a85c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.062136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193cbb2-30bb-423a-9453-ef98bdc14b2a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.081127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.081954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.082240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.082551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.093184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:16.097929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.098781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.099148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.099438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.100633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-da765237-a8ee-470e-aefd-b3619e7a4c39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:16.101449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:16.102252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:16.103318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:16.104254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.104330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 56/108] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:16.104330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a451-8a43-4a2e-b7bb-9360ad960b98 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.104330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a451-8a43-4a2e-b7bb-9360ad960b98 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.104501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a451-8a43-4a2e-b7bb-9360ad960b98 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.110294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 57/109] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:16.110294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1a451-8a43-4a2e-b7bb-9360ad960b98 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.110614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 53/110] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.110614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.184867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.216837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71b171-adae-4170-83f4-d399e2d6f3ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.218699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1a451-8a43-4a2e-b7bb-9360ad960b98 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.248009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.252570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.253140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.253427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.253709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.254503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:16.254818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.256218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.256608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.257232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.257600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.257800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.258592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:16.258899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.260589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a8a49-6b94-4e97-96dc-23c6bca25038 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.264814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 54/111] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.267165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.274854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 58/112] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.274854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.274854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.274854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.289545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:16.290791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:16.356751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 59/113] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:16.357550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.383409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5820221-a493-4f98-98a1-3e57997b1239 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.412082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.417520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.418205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.418524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.418810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.419575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:16.419890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.421289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4423a-2444-4ce9-a214-1523994519a6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.429479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 55/114] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.430813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247ba3a-ba47-4e03-9352-70529565903e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.431105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247ba3a-ba47-4e03-9352-70529565903e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.431387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247ba3a-ba47-4e03-9352-70529565903e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.432579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 56/115] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:16.433279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247ba3a-ba47-4e03-9352-70529565903e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.462876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e355-2329-45b7-b2d8-d51a6c851e83 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.491594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.511859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.512462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.512746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.513024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.513749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.515239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.550522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 60/116] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:01:16 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:16.550522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.568356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247ba3a-ba47-4e03-9352-70529565903e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.610412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.612122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.612411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.612691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] PATH_INFO: `/v3/users/7911ba7f29a84e2c9b67a4a854b4842b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.638634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] RBAC: Authorizing `identity:get_user(user_id=7911ba7f29a84e2c9b67a4a854b4842b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.640315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.642224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 57/117] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/users/7911ba7f29a84e2c9b67a4a854b4842b => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.643271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.707451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516aca1-be12-4784-a825-50e34e98494b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.710473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.761675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.762708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.762936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.763215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.764676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.766176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.773027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 61/118] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.773027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.777067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb57109-c85b-43ba-9af4-58c1212e9c62 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.815087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:16.815991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.817107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.817513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.817882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.830688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d7ae878f-7eaa-4597-a077-8c5594a1461c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:16.838665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:16.839559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:16.840333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 58/119] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:16.841487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.841765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.842049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.863845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:16.865080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:16.894535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426095-f10f-44b4-a58e-be131202fa88 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:16.930424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:16.931727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:16.932364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:16.932840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:16.934289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:16.936180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:16.937968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 62/120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:16.938901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.945590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 59/121] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:16.946833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:16.992595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.054534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb78382-00fd-49d8-ba0f-e66f9531bcb6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.077213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042d713-9000-4956-a211-0464e6e0a8a0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.087423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.089966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.091023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.091314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.091596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.092793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.094272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.096085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 60/122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:17.099196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.115911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.116574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.116862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.117143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.185955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.187466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.208710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c906-bb45-4af3-82a1-c522bec4e66b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.220999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.246700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.247379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.247767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.248043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.248905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:17.249210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.250621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.265836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 61/123] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:17.265836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.299683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 63/124] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:17.299683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.308743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.372639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7baf4c-9036-4e74-b01d-a9e7d565bbf8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.406363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.407254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.407541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.407644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfefc8b-0a45-4c60-ab33-9536dbf0ccf1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.407813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.409262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.410774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.417536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 62/125] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:17.417536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.446178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.447093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.447444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.447732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.458344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:17.461089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:17.466321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:17.467520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:17.468296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 64/126] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:17.469458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.528934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30de3-6ae5-45b7-b11d-f21fb47b1672 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.554887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.563847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.564739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.565026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.565302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/35b22ca699494e3baf04c55a51b0ee59/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.567674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, user_id=35b22ca699494e3baf04c55a51b0ee59, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.568786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.594614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 63/127] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:17 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/35b22ca699494e3baf04c55a51b0ee59/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:17.594614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.633754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388a5d-cc9d-45a8-972a-40e10584ae50 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.691220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.692150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.692532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.692820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.694303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.696002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.698252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 65/128] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:17.699206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.730291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.733786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a929-e168-49c5-964f-dddb5ddc1895 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.790236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.791173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.791474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.791765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.793011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.794513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.796252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 64/129] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:17.797104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.835177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f83dd-bb02-4027-95e1-b54f6be5de01 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:17.870257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:17.871536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:17.872045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:17.872534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/35b22ca699494e3baf04c55a51b0ee59/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:17.875001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, user_id=35b22ca699494e3baf04c55a51b0ee59, role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:17.876672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:17.883195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:17.897301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 66/130] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:17 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/35b22ca699494e3baf04c55a51b0ee59/roles/899a8afb374942c8872cc2929f6b4b1a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:17.899078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:17.966089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd524b-f7e6-4b4d-923b-a534f7dc91e1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.051797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.052507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.052798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.053085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.053864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:18.054178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.055620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.070386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1c5f-67cc-4dbe-b4a9-0fe42b875539 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.076695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 65/131] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:18.077574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:18.123188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.123914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:18.126028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.126824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.127580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.130110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.132781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.135132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 67/132] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:17 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:18.136100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:18.210666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f3ae-1083-484a-93fa-de50174d9064 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.218407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:18.247799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.248450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.248740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.249022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.249865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=7911ba7f29a84e2c9b67a4a854b4842b) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:18.250179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.251716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.258425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 66/133] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:18 2022] GET /identity/v3/role_assignments?user.id=7911ba7f29a84e2c9b67a4a854b4842b&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:18.259110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:18.261512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa0bd-8ec8-4550-b034-9a3e539d07e5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.299705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.301006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.301483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.301950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.303530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.305195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.307132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 68/134] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:18 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:18.313161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:18.368679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:18.378576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510c685-69bc-4edb-9970-81a1b1142a72 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.415002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.415922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.416281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.416568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/769a75081be74879911bd8c4f4cb57da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.419352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, user_id=769a75081be74879911bd8c4f4cb57da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.420176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.436135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 67/135] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:18 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/769a75081be74879911bd8c4f4cb57da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:18.460284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64113-03e2-457b-9e35-aa9101b3d96a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.496116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:18.502464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.503371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.503656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.503986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] PATH_INFO: `/v3/projects/a7de582a100c418393f9fae9b17d0f35` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.505321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] RBAC: Authorizing `identity:get_project(project_id=a7de582a100c418393f9fae9b17d0f35)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.506987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db44be0-355b-41bf-9dd1-81070849698a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.509239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 69/136] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:18 2022] GET /identity/v3/projects/a7de582a100c418393f9fae9b17d0f35 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:18.515067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:18.639465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f207-722b-465f-944e-af9c07d656bd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:18.677167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:18.678074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.678360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.678666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] PATH_INFO: `/v3/projects/a7de582a100c418393f9fae9b17d0f35/users/35b22ca699494e3baf04c55a51b0ee59/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.680693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] RBAC: Authorizing `identity:create_grant(project_id=a7de582a100c418393f9fae9b17d0f35, user_id=35b22ca699494e3baf04c55a51b0ee59, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:18.682170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481250c-df91-4b69-8f62-a8e08e80ff1a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:18.698259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 68/137] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:18 2022] PUT /identity/v3/projects/a7de582a100c418393f9fae9b17d0f35/users/35b22ca699494e3baf04c55a51b0ee59/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:18.897184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644e68-3f47-4d56-8dc2-289cac319b7b None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.897519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644e68-3f47-4d56-8dc2-289cac319b7b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.897692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de644e68-3f47-4d56-8dc2-289cac319b7b None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.899140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 70/138] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:18.904891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:18.905196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:18.905481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:18.928299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:18.929507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:18.940176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:19.036866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 69/139] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:19.084117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de644e68-3f47-4d56-8dc2-289cac319b7b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:19.213040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de644e68-3f47-4d56-8dc2-289cac319b7b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:19.255032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:19.256352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:19.256856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:19.257339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:19.258931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:19.260940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aecf3f-b638-4f28-914b-45b9042e9891 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:19.263156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 71/140] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:19.268799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:19.421387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e25891-8c02-4a3c-927e-7cf213d610d4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:19.465808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:19.466855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:19.467354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:19.467846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:19.468758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:19.470427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55c25b-7e1d-443f-86bf-39fa48260a71 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:19.489458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:19.494916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 70/141] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:19 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:20.732302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a0d4-42a7-4ebf-b1cc-e6763d8815e1 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:20.733149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a0d4-42a7-4ebf-b1cc-e6763d8815e1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:20.733591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68a0d4-42a7-4ebf-b1cc-e6763d8815e1 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:20.735166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 72/142] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:20.741103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6ccf-2de0-4c85-8a93-4f75339f0cbf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:20.741594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6ccf-2de0-4c85-8a93-4f75339f0cbf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:20.741674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6ccf-2de0-4c85-8a93-4f75339f0cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:20.763416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82ee6ccf-2de0-4c85-8a93-4f75339f0cbf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:20.764633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82ee6ccf-2de0-4c85-8a93-4f75339f0cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:20.851011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 71/143] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:20.886987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68a0d4-42a7-4ebf-b1cc-e6763d8815e1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:20.919901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16107d48-77fd-419c-a089-d4398afe74da None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:20.920240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16107d48-77fd-419c-a089-d4398afe74da None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:20.920467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16107d48-77fd-419c-a089-d4398afe74da None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:20.921982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 72/144] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:20.931861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36af92-9196-4514-9ab3-0a9ddd325240 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:20.932173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36af92-9196-4514-9ab3-0a9ddd325240 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:20.932420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36af92-9196-4514-9ab3-0a9ddd325240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:20.958236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fd36af92-9196-4514-9ab3-0a9ddd325240 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:20.959495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fd36af92-9196-4514-9ab3-0a9ddd325240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:21.049703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:21.088815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68a0d4-42a7-4ebf-b1cc-e6763d8815e1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.125994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 73/145] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:21.128055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588154ff-ac40-4cc0-aec1-35676731ddb7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.128553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588154ff-ac40-4cc0-aec1-35676731ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.129017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588154ff-ac40-4cc0-aec1-35676731ddb7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.130825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 74/146] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:21.136662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.137248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.137716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.143512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.145038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.145038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.145038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.156036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:21.157117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:21.157757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:21.166500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:21.167467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:21.168210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 73/147] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:21.168826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.281435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee047a-63b1-4f13-9e32-0f0e2d9c4c1c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.286101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:21.304610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 75/148] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:21.305469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.309314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:21.317075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.317911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.318201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.318530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.329525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f7fbc4e3-bfa4-43f0-8820-c9b81850a7b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:21.339058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:21.339939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:21.340902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 74/149] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:21.341791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440d78-bf06-4d9f-a33e-f3edfecaffc0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.341869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440d78-bf06-4d9f-a33e-f3edfecaffc0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.342125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440d78-bf06-4d9f-a33e-f3edfecaffc0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.347406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 75/150] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:21.347552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440d78-bf06-4d9f-a33e-f3edfecaffc0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.472172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe686-9864-4159-a017-0925e0405005 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.525405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440d78-bf06-4d9f-a33e-f3edfecaffc0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.551996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.553149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.553736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.554213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.555197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:21.555705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:21.557307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2db28-c3cd-4daa-b50e-b88266ead5d0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:21.575612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 76/151] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:21.577334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.577959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.578460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.604670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.605359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.605644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.605921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.606712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:21.607022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:21.608411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:21.609302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:21.610870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:21.637041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 76/152] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:21.637977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.717560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:21.723753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:21.769148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 77/153] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:21.769919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.806418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d659-06f8-42c9-abf5-c5fbb36a85d9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.873913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.874946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.875232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.875510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.896357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:21.897331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46ac6b-f297-4a54-820e-2b49e38cbcca None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:21.904563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:21.905417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:21.906170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 77/154] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:21.916228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:21.940958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:21.942219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:21.942727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:21.943204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:21.944650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:21.946297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:21.948161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 78/155] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:21.949112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.006807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.057825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5ff40-3651-45a8-93e5-465707296c9d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.092769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.093961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.094476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.095034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.107203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:22.115527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8db53-fa12-4892-9951-cf76a819b71c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:22.116133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:22.116974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:22.117726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:22.120169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 79/156] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:22.121153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.133889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.176036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.176991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.177275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.177551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.181188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:22.182857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:22.184364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 78/157] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:21 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:22.185132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.230798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce417030-e2e4-4b19-a17b-19bde50716a2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.243228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.266702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.267961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.268512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.268993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.269955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:22.270505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:22.272145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:22.281161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 80/158] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:22.282298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.374621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1eaf7-b7ed-46bd-9a6e-34ceab345eeb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.396139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c56443-df2f-424e-b0a8-90dc6e790cb5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.431005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.432031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.432504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.432990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.433282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.433955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.434242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.434553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.434794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:22.435279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:22.436842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:22.446175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 81/159] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:22.447327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.487803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.531747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.557652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4122a9-29e2-49db-a76d-da479e00b06d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:22.575775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:22.577265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:22.598874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:22.599851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:22.600321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:22.600781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:22.727044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:22.728532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:22.770241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 79/160] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:22.771121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.798897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.859472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 82/161] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:22.860743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:22.945507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:22.981746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a008f0-62b5-4c15-b69b-bf8e7fea3711 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.025130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b52f1c-1e1e-492e-8f83-31c4d42fa996 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.052269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.053079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.053364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.053633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] PATH_INFO: `/v3/users/7911ba7f29a84e2c9b67a4a854b4842b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.055659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] RBAC: Authorizing `identity:get_user(user_id=7911ba7f29a84e2c9b67a4a854b4842b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.057120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364412f-d525-4a46-9041-3a940d3f2499 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.058782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 80/162] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/users/7911ba7f29a84e2c9b67a4a854b4842b => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.059938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95987eff-7c73-424e-9046-1948010a71f4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.060214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95987eff-7c73-424e-9046-1948010a71f4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.060486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95987eff-7c73-424e-9046-1948010a71f4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.061684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 81/163] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:23.062360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95987eff-7c73-424e-9046-1948010a71f4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.130593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.131482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.131759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.132028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] PATH_INFO: `/v3/users/769a75081be74879911bd8c4f4cb57da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.133824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] RBAC: Authorizing `identity:get_user(user_id=769a75081be74879911bd8c4f4cb57da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.135740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.137376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 83/164] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:22 2022] GET /identity/v3/users/769a75081be74879911bd8c4f4cb57da => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.139525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.150861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.226425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95987eff-7c73-424e-9046-1948010a71f4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.271033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7999be-9404-4fb4-aa25-da44a6c425bb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.271741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.273226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.274492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.275147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.276638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.278222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97b951-b34a-42c6-bcec-ae39cd513848 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.280582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 82/165] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.283303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.283944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.284496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.306017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:23.307650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:23.307790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.308778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.309161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.309504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.310864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.312549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.314597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 84/166] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.315625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.320599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.405413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 83/167] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:23.406786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.446334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.495173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a657-301e-4782-9492-ba50a548d5e2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.526670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83bc89-ec47-417a-b32b-04aa199df7fb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.543640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.544772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.545217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.545762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.547502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.550292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d497231-8994-4f0f-bc9b-6c4ac423c940 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.552638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 85/168] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.554113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645520-3ad2-4021-9deb-86005d168f2b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.554555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645520-3ad2-4021-9deb-86005d168f2b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.554982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645520-3ad2-4021-9deb-86005d168f2b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.556515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 86/169] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:23.557460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0645520-3ad2-4021-9deb-86005d168f2b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.566397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.567639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.567639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.568057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.587032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.651084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.652781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.705000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0645520-3ad2-4021-9deb-86005d168f2b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.752146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.753258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.753769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.754238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.756711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:23.757266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.761343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.762254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 84/170] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:23.763116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.769398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 87/171] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=d4760ffd4761422abfabd45bc0880415 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.770763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:23.774975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.823988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:23.888115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bf37-363b-47b4-aebc-8df4f225cedf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:23.926688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:23.927419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.927946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.928422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.929457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:23.929951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:23.931919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3723e-d318-4e1c-aaec-eb09a9e5780b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:23.939473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 85/172] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:23.941172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:23.941711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:23.942290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:23.962630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:23.964797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:24.027532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff79481-fa09-411b-ad63-78a7b8994167 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.079770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 86/173] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:24.081317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.081937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.083051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.083740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.084217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.098734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:24.107101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:24.108310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:24.109265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:24.111851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 88/174] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:24.118264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.129191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.187952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.209241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48200e-2d2c-43d4-bc37-de96364a7594 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.235943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f38f29-26c5-4cba-8965-80fd83ac4650 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.249169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.250102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.250389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.250689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/7911ba7f29a84e2c9b67a4a854b4842b/roles/c024d0f9c15d42f080c58d0cd5fc6e77` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.252708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, user_id=7911ba7f29a84e2c9b67a4a854b4842b, role_id=c024d0f9c15d42f080c58d0cd5fc6e77)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.254160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd09ac-2e33-4a10-8114-db16cbf8e50e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.266981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 87/175] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:24 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/users/7911ba7f29a84e2c9b67a4a854b4842b/roles/c024d0f9c15d42f080c58d0cd5fc6e77 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:24.268349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057378fa-6c37-4f39-97f7-853f9f5ef2ff None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.268590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057378fa-6c37-4f39-97f7-853f9f5ef2ff None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.268863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057378fa-6c37-4f39-97f7-853f9f5ef2ff None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.270141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 88/176] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:24.271353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd72f7-e8f0-4c8e-a7bd-967b459ae6c4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.271785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd72f7-e8f0-4c8e-a7bd-967b459ae6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.272104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd72f7-e8f0-4c8e-a7bd-967b459ae6c4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.273455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 89/177] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:24.274179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd72f7-e8f0-4c8e-a7bd-967b459ae6c4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.275079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.275964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.276244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.276515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.278328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.279847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.281830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 89/178] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.283611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.398447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.403881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd72f7-e8f0-4c8e-a7bd-967b459ae6c4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.405632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85221b-2510-4d9a-a7f2-4d5f5018eea8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.442006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.449556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.450244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.450624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.450737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.450917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.451431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.451660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:24.451763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.451972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.452067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.453351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3aa7-c321-4654-9836-574acdb3f93c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.462867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:24.463035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 90/179] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.463888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.464170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.464450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ce8f49c4-64ee-4ef4-862d-2cc767ded1f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:24.470567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:24.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:24.472616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 90/180] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:24.473426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.473426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.473572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.488332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:24.489932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:24.490501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:24.491646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:24.562437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 91/181] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:24.562437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.572613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 91/182] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:24.573664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.678908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.688656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6e7bc-7cdd-4570-819c-70baa3b73269 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.692270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce681d7-4dc4-4e7d-b3be-7c2b39b0f9d7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.713618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.726913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.727810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.728094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.728371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] PATH_INFO: `/v3/users/35b22ca699494e3baf04c55a51b0ee59` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.729561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] RBAC: Authorizing `identity:get_user(user_id=35b22ca699494e3baf04c55a51b0ee59)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.730272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.730841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.731072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.731168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.731388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.732234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=769a75081be74879911bd8c4f4cb57da) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:24.732547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.732916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 92/183] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/users/35b22ca699494e3baf04c55a51b0ee59 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.733602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.733981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.742355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 92/184] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/role_assignments?user.id=769a75081be74879911bd8c4f4cb57da&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.743180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.863031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77fc2-d82b-431d-b3cc-bb1e6509d384 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.878701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537cada-a855-44f5-ad45-1ff21be53aa2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:24.904151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.905031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.905382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.905726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.906694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:24.907073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.908963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.918518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:24.919703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:24.920162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:24.920599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:24.921985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:24.922614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 93/185] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.923704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.924227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:24.926129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 93/186] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:24.931365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:24.956579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:24.986563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.047291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c98290-bb8c-4ab4-a438-bf825971acc7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.056376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f9e04-c631-4c27-8652-bdb12baf103d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.082596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.083417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.083700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.083976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.094156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:25.096919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.097732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.098171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.098611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:25.102274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:25.103306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:25.104078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:25.108388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 94/187] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:25.108995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.169545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.171101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.191486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.219280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d0c5c-4288-4c4c-9491-b8963fd38531 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.233557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.256879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.257767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.258047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.258316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.272955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.274438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.276319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 95/188] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:25.276856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.284088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 94/189] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:25.284774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.385222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e650e25-26e5-4635-a84a-e1c1d15b87e2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.387724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c18fa-12d7-4133-9e54-eab098463d69 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.411966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.420982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.421655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.421937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.422210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.422958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.424434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.430056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.431083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.431533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.432366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.433731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:25.434852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.436812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.445059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 96/190] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:25 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:25.446041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.453365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:25.454650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.492756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.565869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ddad-99aa-41cd-b101-d969788b808a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.583305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba36bb-fb94-4b39-8d86-e368629d6d35 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.615423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.616321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.616606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.616884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] PATH_INFO: `/v3/projects/a7de582a100c418393f9fae9b17d0f35` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.618424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] RBAC: Authorizing `identity:get_project(project_id=a7de582a100c418393f9fae9b17d0f35)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.619864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.621824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 96/192] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/projects/a7de582a100c418393f9fae9b17d0f35 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:25.622710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.632651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.633316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.633601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.633880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.712380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.724797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.726760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.764230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:25.791426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d683d-780e-4d57-80da-456d1db69db2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:25.842523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:25.843302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:25.843588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:25.843868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:25.845113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:25.846768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:25.848605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 97/193] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:25.849179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.868673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 97/194] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:25.869614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:25.974647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d958df-ab1b-48f9-ae16-c27fa4fe2a41 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.005630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.021263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e961b0-ada8-459e-98b2-4238a226f333 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.037723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.038417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.038707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.038983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.039838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=a7de582a100c418393f9fae9b17d0f35, user.id=35b22ca699494e3baf04c55a51b0ee59) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:26.040155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.041615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.050933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 98/195] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/role_assignments?user.id=35b22ca699494e3baf04c55a51b0ee59&scope.project.id=a7de582a100c418393f9fae9b17d0f35&role.id=d4760ffd4761422abfabd45bc0880415 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:26.051585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:26.086199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.099448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.100828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.101303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.101770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.195585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.197093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.310556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 98/196] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:26.317060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:26.339147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d8b3b-4b59-4b5c-84bf-d0bd67c61181 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.358739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.402614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.404179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.404179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.404179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.405330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.406817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.409114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 99/197] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:26.415350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:26.464314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04431-0f72-40e5-b57d-5602f66f0a0c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.498279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.510504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.512018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.512444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.512985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.515103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.517051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e72ff-fc71-4596-ace2-59a702cd21fb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.519925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 99/198] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:26.521393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8730-db46-4a48-9feb-416cf4c6b779 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.521805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8730-db46-4a48-9feb-416cf4c6b779 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.522203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8730-db46-4a48-9feb-416cf4c6b779 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.523599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 100/199] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:26.531291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.531800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.532211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.557862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:26.560063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:26.567506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3cc0c-b35d-4bb6-ab1a-f6b5e8b48674 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.610399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.611533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.611870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.614881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.615804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=77c5b166d66341cd952aa298f7e8011a) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:26.616129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.617579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.625721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 100/200] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/role_assignments?user.id=77c5b166d66341cd952aa298f7e8011a&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:26.625878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:26.674701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 101/201] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:26.720257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:26.762953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.794816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ca7a-9571-406a-81b8-aecdca07574a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.825921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:26.857867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.858934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.859415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.859906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.860997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=77c5b166d66341cd952aa298f7e8011a) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:26.861731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:26.863555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:26.870707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 101/202] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/role_assignments?user.id=77c5b166d66341cd952aa298f7e8011a&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:26.874556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43165-f5f6-475b-a861-df46faccfcc9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:26.912660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:26.913534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:26.913814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:26.914081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:26.926109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:26.934016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:26.934914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:26.935702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:26.938552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 102/203] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:26.941511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:27.057486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dcbf1-9b39-4ddf-b35a-fa055dac4118 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:27.098211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:27.098863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:27.099150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:27.099431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:27.100214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:27.100530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:27.102060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:27.112178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 102/204] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:27.123574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:27.225733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:27.241498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40b1fc-6931-45eb-b0b6-a864f7702354 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:27.275766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:27.276662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:27.276941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:27.277212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] PATH_INFO: `/v3/users/7911ba7f29a84e2c9b67a4a854b4842b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:27.278502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] RBAC: Authorizing `identity:get_user(user_id=7911ba7f29a84e2c9b67a4a854b4842b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:27.280016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:27.281651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 103/205] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:27 2022] GET /identity/v3/users/7911ba7f29a84e2c9b67a4a854b4842b => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:27.287509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:27.362072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:27.449088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83689e-adb9-403c-ab0d-4b20faf5b979 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:27.492156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:27.492816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:27.493099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:27.493371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:27.583494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:27.584979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:27.644850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:27.735609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 103/206] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:27.743484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:27.866272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465ce0-522c-473a-afcd-78981d06d7cb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:27.900520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:27.901315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:27.901595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:27.901869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:27.907089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:27.908850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070ca64-b6ff-491d-9fb8-fb152222e74f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:27.910607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 104/207] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:27 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:27.916384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.083171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7d507-5d13-4070-96a7-3963ffa2bed1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf53f3c-9069-412e-ae81-22e32403e682 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.085907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf53f3c-9069-412e-ae81-22e32403e682 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.086182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf53f3c-9069-412e-ae81-22e32403e682 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.087902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 105/208] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:28.095778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc254ab-e934-40a0-9177-9228a2975167 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.096144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc254ab-e934-40a0-9177-9228a2975167 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.096649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc254ab-e934-40a0-9177-9228a2975167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.120077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6bc254ab-e934-40a0-9177-9228a2975167 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:28.121262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6bc254ab-e934-40a0-9177-9228a2975167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:28.136066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.151712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.152320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.152607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.152885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.153741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8, user.id=7911ba7f29a84e2c9b67a4a854b4842b) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:28.154058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:28.155553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:28.169552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 104/209] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:27 2022] GET /identity/v3/role_assignments?user.id=7911ba7f29a84e2c9b67a4a854b4842b&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:28.200706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 106/210] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:28.237211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.246497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.378240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8eac-41ff-450c-a808-b337a9aae6cc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.417527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.418714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.419207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.419700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.430984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:28.438426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583135-9d2b-457a-86bc-bf6ffe1fac7e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.438722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583135-9d2b-457a-86bc-bf6ffe1fac7e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.438996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583135-9d2b-457a-86bc-bf6ffe1fac7e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1f3f9e0d-cbd9-4ea7-92d4-1c393668d497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:28.440352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:28.441183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:28.441979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:28.442847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 107/211] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:28.442914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 105/212] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:28.448911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad583135-9d2b-457a-86bc-bf6ffe1fac7e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.453379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.453674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.453958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.472914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:28.474314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:28.553358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.564135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 106/213] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:28.569123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad583135-9d2b-457a-86bc-bf6ffe1fac7e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.592289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.604810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.604810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.604810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.604810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.605312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:28.605504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:28.606926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:28.616467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 108/214] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:28.625632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.648244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.743182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba7332-3203-4d47-8665-6ec58a5e33bc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.750843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34f2f9-f77f-4cac-989a-055f10806e9c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.788631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.789522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.789801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.790073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] PATH_INFO: `/v3/users/a6afd59e66f24800b695589f8b734a22` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.795000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.796101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.796597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.797071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.799873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:28.801972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:28.804662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 107/215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:28.805040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] RBAC: Authorizing `identity:get_user(user_id=a6afd59e66f24800b695589f8b734a22)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:28.810401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:28.810401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 109/216] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:28 2022] GET /identity/v3/users/a6afd59e66f24800b695589f8b734a22 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:28.811499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.817409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:28.879444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.922426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e24a-c97f-4524-b982-fd03c69d0dd0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.922758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:28.959372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cc91-da16-4ffd-8c6d-6b87121a0bdb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:28.970276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:28.972050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:28.972339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:28.972620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:28.973645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:28.975393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:29.000606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.001546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.002167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.002167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.004893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 108/217] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:28 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:29.088945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:29.090681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f70931-1105-46a8-b80c-dfe7357c8326 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:29.168019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:29.202625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 110/218] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:29.209203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.269831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8330b-445e-440d-bbe0-70e887b80b2e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.270110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8330b-445e-440d-bbe0-70e887b80b2e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.271075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8330b-445e-440d-bbe0-70e887b80b2e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.272118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 111/219] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:01:29.280137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595a5c-8bb9-47cc-8168-3775770952bf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.280513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595a5c-8bb9-47cc-8168-3775770952bf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.280797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36595a5c-8bb9-47cc-8168-3775770952bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.300486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-36595a5c-8bb9-47cc-8168-3775770952bf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:29.301699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36595a5c-8bb9-47cc-8168-3775770952bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:29.332681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfc80a-0f17-4d4e-ab9c-cd714fe566eb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:29.356166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:29.379400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.380297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.380583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.380866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.383297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:29.385495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c16519-5f2f-490d-aae1-69358fffec4c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:29.387460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 109/220] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:29.388797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8ab9-4ec0-475f-8067-986d311f2c12 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.389084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8ab9-4ec0-475f-8067-986d311f2c12 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.389368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8ab9-4ec0-475f-8067-986d311f2c12 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.391118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 110/221] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:29.394206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f8ab9-4ec0-475f-8067-986d311f2c12 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.397581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 112/222] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:29.401684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.401684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.401684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.419943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:29.421596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:29.515105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f8ab9-4ec0-475f-8067-986d311f2c12 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:29.518202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 113/223] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:29.520227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.556766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.557832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:29.558336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.559148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.559757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.560943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=a6afd59e66f24800b695589f8b734a22) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:29.561459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:29.563140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:29.569768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 111/224] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/role_assignments?user.id=a6afd59e66f24800b695589f8b734a22&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=d4760ffd4761422abfabd45bc0880415 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:29.575660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.647102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5d843-3c99-4375-9510-d36ee4a7a5a4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:29.691443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.692310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.692595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.692872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.722546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:29.725333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:29.737215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:29.738216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:29.739086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:29.739688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 114/225] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:29.746801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.820117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6cb47-365a-4d77-adf4-7037113e442e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:29.861095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.861095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.861095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.861095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.877866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400b36-26f8-4ed3-8181-ab7885775e96 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:29.884335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:29.885518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:29.901494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:29.902541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:29.903347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 112/226] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:29.910641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:29.913414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:29.914358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:29.914842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:29.915304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:29.916239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:29.916686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:29.918301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:29.928038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 115/227] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:29.933826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.048396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345611f3-ddc2-4822-8aed-9bcd9e26addf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.052546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f38f-fce6-4293-8714-5eda5be8b231 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.066856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:30.108317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.109127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.109410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.109688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.110876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.112164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.112448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.112727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.113505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:30.114437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.115930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.116340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.117725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.120663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 116/228] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:30.127128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.134142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 113/229] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:30.140268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.162745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:30.304415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e26ac-ee2a-406a-b08e-22bf753f5844 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.325080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f5c6-ebd3-4014-b3cc-6d78e94005bc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.367182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.368004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.368837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.369120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.371191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.373789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.378551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 117/230] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:30 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:30.383701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.390498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.391455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.391745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.392028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.394113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.395642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.397465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 114/231] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:30 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:30.403529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.478749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:30.505015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:30.528291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff961-8caa-4bfc-ab81-6188775a9a31 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.575535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc123b-e510-4a02-999d-a8a0e2aaa228 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.608730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.609609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.609894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.610171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.611442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.613065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.613793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.614802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 115/232] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:30 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:30.615056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.615363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.615645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/users/77c5b166d66341cd952aa298f7e8011a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.626463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, user_id=77c5b166d66341cd952aa298f7e8011a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.626463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af228ea-3980-438d-877c-ebe01bd6910c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.627884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:30.646497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 118/233] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:30 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/users/77c5b166d66341cd952aa298f7e8011a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:01:30.782799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005e8d-f864-4282-bfe5-ae98e9c1614a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:30.801059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:30.832120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:30.833330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:30.833617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:30.833892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/users/77c5b166d66341cd952aa298f7e8011a/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:30.836163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, user_id=77c5b166d66341cd952aa298f7e8011a, role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:30.837641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b24fca-5c5c-4776-9f25-e93803cd1ea2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:30.862588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 116/234] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:30 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/users/77c5b166d66341cd952aa298f7e8011a/roles/899a8afb374942c8872cc2929f6b4b1a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:31.407037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0950c-a190-4e90-9728-dcfeb03181a6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.407558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0950c-a190-4e90-9728-dcfeb03181a6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.407631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0950c-a190-4e90-9728-dcfeb03181a6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.409864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 119/235] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:31.415445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.416303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.416786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.443426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:31.444956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:31.526240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 117/236] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:31.548393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3910-cbc8-4999-82c9-369c8d2f824c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.548948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3910-cbc8-4999-82c9-369c8d2f824c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.549229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3910-cbc8-4999-82c9-369c8d2f824c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.550195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:31.551122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 120/237] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:31.556032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.556324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.556605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.591622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:31.594752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:31.711866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:31.728305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65538e-06b8-4428-b499-a7a151231050 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:31.742672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 121/238] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:31.767001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:31.780570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:31.781225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.781506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.781836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.796872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:31.804919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:31.805822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:31.810464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 118/239] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:31.812284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:31.908031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:31.916024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe15ca-96d0-4a39-9945-c7da7ce16600 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:31.947104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57dc3b-bfb3-4839-b9e1-fe677fcf5956 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:31.951902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:31.957404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:31.966485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 119/240] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:31.971679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:31.994946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.011493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.012315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.012603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.012888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.014072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.015582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.018317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 122/241] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:32.027292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.080555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1cea-f7d9-40ed-9cc7-4e78b5464902 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.118590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.119711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.120406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.120705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] PATH_INFO: `/v3/users/372fdbb565c046898241245fd2fdcbf7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.141015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba1b5-177c-42db-9c58-222e2d83faa6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.141210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] RBAC: Authorizing `identity:get_user(user_id=372fdbb565c046898241245fd2fdcbf7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.143125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7211a-108d-4a7e-8e25-6304f071e936 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.150835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 120/242] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:31 2022] GET /identity/v3/users/372fdbb565c046898241245fd2fdcbf7 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:32.152935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a6ec-2c3d-4098-8e6c-06439bed2e1a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.153326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a6ec-2c3d-4098-8e6c-06439bed2e1a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.153716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a6ec-2c3d-4098-8e6c-06439bed2e1a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.155884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 121/243] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:32.156795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52a6ec-2c3d-4098-8e6c-06439bed2e1a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.195191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.196676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.197207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.197756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.198881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.200666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e3439-0ce2-4e7c-8389-6ca7b54fd956 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.225797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 123/244] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:32 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:32.228212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.228686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.229112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.236803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.255230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:32.257909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.258593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:32.286311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52a6ec-2c3d-4098-8e6c-06439bed2e1a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.329747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.331434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.331886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.332590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.346417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 124/245] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:32.381098 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.417526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.420055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.514266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.548792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 122/246] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:32.555579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.565787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be9a4a-7be9-461a-a86b-12187d5e21d0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.605199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.612348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.613839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.614406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.615239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.638209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:32.646419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:32.647430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:32.648680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 125/247] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:32.653640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.702091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595bee6-11ec-4828-a3c2-917fa9016332 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.746396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.747545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.748314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.748783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.750502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.753042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.755625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 123/248] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:32.761815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.783136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd9e04-4ce9-4808-a96a-cbf2ebf79167 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.821520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.829478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.830239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.830561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.830838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.831610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:32.831922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.834096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.844912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 126/249] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:32.851037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:32.891603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d2bf3-0890-4ef0-8c63-8660a0d20eeb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:32.927218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:32.945231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:32.946337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:32.946811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:32.947196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:32.948313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=372fdbb565c046898241245fd2fdcbf7) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:32.949536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:32.951135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:32.960000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 124/250] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/role_assignments?user.id=372fdbb565c046898241245fd2fdcbf7&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:32.993143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e4672-1629-4aa8-b2c3-b9b4c494ee2c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:33.034354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:33.035282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.035568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.035888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] PATH_INFO: `/v3/users/a6afd59e66f24800b695589f8b734a22` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.038147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] RBAC: Authorizing `identity:get_user(user_id=a6afd59e66f24800b695589f8b734a22)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:33.039686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1406b8-ee28-4476-8af5-a64eddca1031 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:33.041870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 127/251] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:32 2022] GET /identity/v3/users/a6afd59e66f24800b695589f8b734a22 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:33.047416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:33.168515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeff4a9-3170-46e2-b44c-a3656fc3502d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:33.180305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:33.210105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:33.211032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.211310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.211579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.212774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:33.214245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:33.216002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 125/252] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:33 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:33.216576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597bad-af72-476f-9c53-3aaa48392bd9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.216935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597bad-af72-476f-9c53-3aaa48392bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.217207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597bad-af72-476f-9c53-3aaa48392bd9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.219700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 128/253] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:33.221899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:33.224308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.224602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.225010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.260050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:33.261404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:33.331090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac55ce0-8f52-428a-b1d6-2625986ddbbd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:33.367653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:33.368938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.369424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.369913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/users/a6afd59e66f24800b695589f8b734a22/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.372234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, user_id=a6afd59e66f24800b695589f8b734a22, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:33.373924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:33.390468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 126/254] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:33 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/users/a6afd59e66f24800b695589f8b734a22/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:33.409807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 129/255] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:33.469278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:33.518732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:33.587646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ae832-d367-4204-9cca-b546d2277b43 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:33.625057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:33.626254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.626954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.628014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.640076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7bd4cb73-a89f-4256-9ec7-cf8cce8bcb23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:33.648888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:33.649822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:33.651280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 127/256] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:33.660188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:33.740147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:33.782480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2b6-cb74-4291-a2ca-bbb2fb08c50a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.782818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2b6-cb74-4291-a2ca-bbb2fb08c50a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.783083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2b6-cb74-4291-a2ca-bbb2fb08c50a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.785594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 128/257] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:33.791040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.791326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.791606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.798627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77060d7-dd57-4dcf-98d3-3194da700ad1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:33.813364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:33.814697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:33.879250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:33.879861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:33.880194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:33.880476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:33.881242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:33.881550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:33.882982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:33.895948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 129/258] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:33.898094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 130/259] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:33.907466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:33.938846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:33.939998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:34.118655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d821d-0be4-4038-8122-c8b8e8ec6680 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.142859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:34.156469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7e68d-2031-4d4f-9da8-a03980aa0ebf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.193178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:34.194008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:34.194293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:34.194594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:34.221356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:34.244904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:34.245845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:34.246751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:34.248493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 131/260] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:34.253108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:34.254009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:34.254294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:34.254602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:34.256463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:34.257217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:34.258714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:34.264676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 130/261] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:33 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:34.271367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:34.456321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:34.462128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f9d34-bc88-4217-9a6e-6491e32707a0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.466921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91855f-591a-4b69-bba4-7022a5dbdb60 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.522552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:34.523245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:34.523530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:34.523808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:34.524578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:34.524892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:34.526309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:34.531000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:34.531688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:34.531972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:34.532251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:34.550416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 132/262] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:34.555409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:34.610808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:34.700112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012b98-7f28-4d41-a04d-df2ee05f85cb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.739506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:34.741309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:34.767647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:34.768553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:34.768839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:34.769119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] PATH_INFO: `/v3/users/77c5b166d66341cd952aa298f7e8011a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:34.775029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] RBAC: Authorizing `identity:get_user(user_id=77c5b166d66341cd952aa298f7e8011a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:34.776720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:34.778834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 133/263] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:34 2022] GET /identity/v3/users/77c5b166d66341cd952aa298f7e8011a => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:34.784302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:34.947972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f6dc0-a5b3-45c0-bb7f-327c193e37d7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:34.950627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:34.951394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:34.967169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 131/264] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:34.972640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:35.007765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:35.008598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.008895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.009172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.152913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:35.154504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:35.240570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde14fd-fe32-40f8-890b-0ebd6a683fbe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:35.324380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 134/265] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:35.329383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:35.338778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:35.347824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:35.348727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.349009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.349285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.351593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:35.353061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:35.355325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 132/266] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:34 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:35.372275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:35.486251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee8fb8-5053-466a-913a-d4844fe6dd22 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:35.520738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:35.521610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.521890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.522169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.522678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:35.523361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:35.524884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:35.526847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 135/267] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:35 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:35.532389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:35.603128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:35.612341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce685c-1ca7-44ee-bc49-e6a66716b5c1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:35.633689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f0f09-64e1-4c53-a463-0d68e2d93827 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:35.674090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:35.675024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.675313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.675990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.677101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=77c5b166d66341cd952aa298f7e8011a) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:35.677421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:35.679194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616d3b-22c5-4ee2-b915-1b6d0f1a0bf6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:35.680426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:35.681154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.681499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.681836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.682781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=77c5b166d66341cd952aa298f7e8011a) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:35.683153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:35.684641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca1541-d10c-4144-9523-f620ce01328c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:35.686032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 136/268] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:35 2022] GET /identity/v3/role_assignments?user.id=77c5b166d66341cd952aa298f7e8011a&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:35.690485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5e3e-5233-46fb-8a4a-0a77916e9a83 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.690884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5e3e-5233-46fb-8a4a-0a77916e9a83 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.691168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5e3e-5233-46fb-8a4a-0a77916e9a83 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.692552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 137/269] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:35.693762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fba07-b938-4c61-80a2-df7f483228b6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.694044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fba07-b938-4c61-80a2-df7f483228b6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.694317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fba07-b938-4c61-80a2-df7f483228b6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.696764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 138/270] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:35.703351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 133/271] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:35 2022] GET /identity/v3/role_assignments?user.id=77c5b166d66341cd952aa298f7e8011a&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:35.704883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.705395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.705468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.708185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:35.708470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:35.708747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:35.738938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:35.740164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:35.743445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:35.744645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:35.869894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 139/272] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:35.870891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 134/273] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:35.895919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:35.912683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.015784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.020185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.071835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d504f6-87b4-4036-a216-e4ba3e83b285 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.072057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b968e3-332e-40b1-9a3c-e3f16fc299b8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.124670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.125488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.125777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.126056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.128247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.129355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.129643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.129924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.133512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:36.135554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:36.137257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 140/274] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:36.140045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:36.142757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:36.148827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:36.149718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:36.150547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:36.151295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 135/275] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:36.157898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.276591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd368ec-c64f-4d0f-8768-7bc895b504e9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.288924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9498e-3802-49e7-b8a3-f9c9e48a6876 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.312144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.317362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.318142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.318480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.318765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.319504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:36.320958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15ffcb-9608-4376-951a-c6f4bd927026 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:36.329520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.332652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.333693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.333994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.334475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.335820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:36.336176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:36.337619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:36.347956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 136/276] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:36.353607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.359267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 141/277] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:01:36 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:36.360648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664ccc-ac70-495e-8281-23429e7418b5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.361009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664ccc-ac70-495e-8281-23429e7418b5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.361286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664ccc-ac70-495e-8281-23429e7418b5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.362895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 142/278] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:36.380848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.381212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.381588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.408399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:36.409858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:36.490766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14587ac4-a02c-43ad-aacf-c4dcc1a54b35 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.511729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 143/279] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:36.534683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.538827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.540327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.540794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.541249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] PATH_INFO: `/v3/users/69815719db2248a7b1f5da5b4a7e9e92` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.557503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] RBAC: Authorizing `identity:get_user(user_id=69815719db2248a7b1f5da5b4a7e9e92)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:36.559021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:36.561187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 137/280] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/users/69815719db2248a7b1f5da5b4a7e9e92 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:36.566797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.614342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.658000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2b3c8-5f09-42cb-b32b-9adfe9a9b0cc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.682753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.695237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ba48-d9d4-42f5-b021-bdbfeb338191 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.700932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.701749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.702025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.702299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.730030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:36.732385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:36.733058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.733345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.733625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.734607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, user.id=69815719db2248a7b1f5da5b4a7e9e92, scope.system=all) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:36.734818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:36.736324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-55620f05-4dd7-4629-ad9e-8b0ac9fbe4b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:36.739159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:36.740020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:36.740812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 144/281] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:36.741757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbe85-ed46-4e0d-816e-bec78aed9645 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.741918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbe85-ed46-4e0d-816e-bec78aed9645 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.742283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbe85-ed46-4e0d-816e-bec78aed9645 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.744386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 138/282] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/role_assignments?user.id=69815719db2248a7b1f5da5b4a7e9e92&scope.system=all&role.id=d4760ffd4761422abfabd45bc0880415 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:36.747953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 145/283] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:36.750421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:36.756655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:36.756947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:36.757238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:36.809764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:36.812085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:36.928775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265046-0898-446d-95bb-a9c843316e02 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:36.933123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:36.961019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 146/284] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:36.985457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.000239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.000925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.001201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.001520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.002279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:37.002624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.004011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.021776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 139/285] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:37.027670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.154451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4e2a-cf5e-43ba-9fa8-7554d1919a4d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.181609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1c7a-ccf3-4c93-8800-b4ed48638959 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.188742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.189989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.190521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.190967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] PATH_INFO: `/v3/users/372fdbb565c046898241245fd2fdcbf7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.192363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] RBAC: Authorizing `identity:get_user(user_id=372fdbb565c046898241245fd2fdcbf7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.193988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.195860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 140/286] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:37 2022] GET /identity/v3/users/372fdbb565c046898241245fd2fdcbf7 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:37.203804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:37.206705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.242200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.243410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.243890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.244595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.262827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:37.277394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:37.288845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:37.289739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:37.290559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:37.300146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 147/287] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:37.302007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.436341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82591e9-ab16-4456-889a-876afd52148b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.440948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9b44c-4adc-4c20-bd4b-3df160baba50 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.487559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.488599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.490788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.491267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.492220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:37.493985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.495612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.516311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 148/288] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:37.517370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.518560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.519098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.520134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.520998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.525380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.527077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.528985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 141/289] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:37 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:37.534844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.558064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:37.662549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:37.666925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695a9f-7176-4fcb-bd22-41d87bc34100 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.716432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.717698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.718205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.718704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/users/372fdbb565c046898241245fd2fdcbf7/roles/c024d0f9c15d42f080c58d0cd5fc6e77` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.721366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, user_id=372fdbb565c046898241245fd2fdcbf7, role_id=c024d0f9c15d42f080c58d0cd5fc6e77)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.723106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.744126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 142/290] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:01:37 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/users/372fdbb565c046898241245fd2fdcbf7/roles/c024d0f9c15d42f080c58d0cd5fc6e77 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:01:37.746697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0fcf-4bd5-43e8-8b81-bf758fd8514e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.800042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.801372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.801864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.802387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] PATH_INFO: `/v3/users/a6afd59e66f24800b695589f8b734a22` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:37.804318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] RBAC: Authorizing `identity:get_user(user_id=a6afd59e66f24800b695589f8b734a22)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:37.806166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:37.813217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 149/291] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:37 2022] GET /identity/v3/users/a6afd59e66f24800b695589f8b734a22 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:37.815509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:37.936827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f4b47-9be5-4916-a57a-5e68a66c7f28 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:37.973718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:37.974302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:37.974615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:37.974892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.004811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:38.058577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:38.060044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:38.185240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 143/292] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 04:01:38.191225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:38.344003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:38.350646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97872f65-3432-41e7-9dbe-05c19820e937 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:38.403107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:38.404006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.404299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.404582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.405852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:38.407347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef7afd-8d79-48e2-8a08-e8b5f42367eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:38.409856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 150/293] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:38 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:38.415839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:38.495806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:38.554202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21dc4-9703-4622-a974-b72004bda34e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:38.593408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:38.594133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.594352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.594648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.595561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=a6afd59e66f24800b695589f8b734a22) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:38.595872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:38.597345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f38ef-d042-4757-8a49-d7ca377ecda2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:38.604187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 144/294] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:38 2022] GET /identity/v3/role_assignments?user.id=a6afd59e66f24800b695589f8b734a22&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=d4760ffd4761422abfabd45bc0880415 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:38.831138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752d17f-1523-46bf-a905-e59054ea3f17 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.831662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752d17f-1523-46bf-a905-e59054ea3f17 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.831734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752d17f-1523-46bf-a905-e59054ea3f17 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.833315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 151/295] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:38.839063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.839551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.839654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.867590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:38.868729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:38.918685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3d34-fa77-4cb4-b8f0-e8c9a10461e6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.918987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3d34-fa77-4cb4-b8f0-e8c9a10461e6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.919254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3d34-fa77-4cb4-b8f0-e8c9a10461e6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.921040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 152/296] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:01:38.935847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5de4-0791-494e-82b4-6f5814ea8f70 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:38.936166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5de4-0791-494e-82b4-6f5814ea8f70 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:38.936408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5de4-0791-494e-82b4-6f5814ea8f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:38.947153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 145/297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:38.960208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5a7e5de4-0791-494e-82b4-6f5814ea8f70 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:38.961400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5a7e5de4-0791-494e-82b4-6f5814ea8f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:38.970500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.038317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.049955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 153/298] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:39.061378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb686ec-aab6-4558-81da-60355dbc3cd2 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.061958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb686ec-aab6-4558-81da-60355dbc3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.062485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb686ec-aab6-4558-81da-60355dbc3cd2 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.064096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 154/299] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:01:39.069445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.069998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.070503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.088162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e79a1c-f1fb-447d-a464-03dca07429c6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.088343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:39.089464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:39.123456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.124190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.124461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.124731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.136079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:39.144124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:39.144975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:39.145770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 146/300] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:39.147385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.165411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 155/301] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:39.166081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.196408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.260622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba589c0-732d-4efe-9ed1-1b87230e8403 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.271322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c761c-2c5a-4d28-9be3-dfa744f1b75d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.275335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.297652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.298634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.299072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.299366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.300539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.302108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.308392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 147/302] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.308392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.311036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.311715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.311993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.312265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.313040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:39.313347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.314970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.323950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 156/303] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.324566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.419042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.421084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189beb43-91bf-46e7-b5c8-7b71f308ba7b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.428375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36d6aa-36e3-48a9-80a4-1fd4544d32be None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.455946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.456733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.457015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.457324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.464024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.464703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.464992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.465270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] PATH_INFO: `/v3/groups` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.465964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.467389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:39.467506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:39.476384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:39.477550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 148/304] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:39.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.490070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 157/305] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Jul 4 04:01:39 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:39.503418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.535845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.627489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db71f3d-5f67-437d-be3d-7db2aa50033e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.644002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a773-6a4b-4ff1-91ef-b02033f61954 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.672831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.673690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.673975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.674253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] PATH_INFO: `/v3/users/500182e9c0774682a5ebebd9f4290eef` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.688535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.689226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] RBAC: Authorizing `identity:get_user(user_id=500182e9c0774682a5ebebd9f4290eef)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.689522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.689805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.690080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.690867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:39.690981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.691187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.692605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.692901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 149/306] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/users/500182e9c0774682a5ebebd9f4290eef => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.704075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.708634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 158/307] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.714524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.730699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.819304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a66f-925b-455b-801e-749d0a168342 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.825977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.830731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42869f0-4692-4ea5-9416-d03df349793a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:39.860477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.861484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.861722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.862004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] PATH_INFO: `/v3/users/69815719db2248a7b1f5da5b4a7e9e92` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.863263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] RBAC: Authorizing `identity:get_user(user_id=69815719db2248a7b1f5da5b4a7e9e92)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.864724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.866827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 159/308] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/users/69815719db2248a7b1f5da5b4a7e9e92 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.872229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:39.876817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:39.877503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:39.877781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:39.878052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:39.878949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, user.id=500182e9c0774682a5ebebd9f4290eef, scope.system=all) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:39.879238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:39.880914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327613-78c7-46b5-8b03-e38993f3fe9e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:39.887375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 150/309] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Jul 4 04:01:39 2022] GET /identity/v3/role_assignments?user.id=500182e9c0774682a5ebebd9f4290eef&scope.system=all&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:39.970558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:39.991319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666e819-c668-415c-bedf-20a2c133a81c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:40.028775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:40.030116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.030671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.031181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] PATH_INFO: `/v3/system/users/69815719db2248a7b1f5da5b4a7e9e92/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.033122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=69815719db2248a7b1f5da5b4a7e9e92, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:40.034863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658af2ee-d659-4157-9b97-4357b4f7aab4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:40.046553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 160/310] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Jul 4 04:01:39 2022] PUT /identity/v3/system/users/69815719db2248a7b1f5da5b4a7e9e92/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:01:40.330816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a335741-c7f6-4d3b-aa62-08ec9e1b126c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.331188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a335741-c7f6-4d3b-aa62-08ec9e1b126c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.331375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a335741-c7f6-4d3b-aa62-08ec9e1b126c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.333020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 151/311] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:40.338486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.339026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.339499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.357146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:40.358703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:40.431467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 161/312] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:40.455426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a335741-c7f6-4d3b-aa62-08ec9e1b126c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:40.558970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a335741-c7f6-4d3b-aa62-08ec9e1b126c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:40.593431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:40.594156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.594463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.594801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.605680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:40.613708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:40.614617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:40.615393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:40.619659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 152/313] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:40.621176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:40.710730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:40.736758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c180229-b296-4c66-94d6-9b78f26b6eda None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:40.772609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:40.773296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.773579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.773862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.774651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:40.774958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:40.776344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:40.785373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 162/314] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:40.790937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:40.818888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:40.898844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c3a94-6e34-48b7-b633-3e5d58b17548 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:40.932764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:40.933700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:40.933987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:40.934264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] PATH_INFO: `/v3/users/372fdbb565c046898241245fd2fdcbf7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:40.935758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] RBAC: Authorizing `identity:get_user(user_id=372fdbb565c046898241245fd2fdcbf7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:40.937318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:40.943546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 153/315] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:40 2022] GET /identity/v3/users/372fdbb565c046898241245fd2fdcbf7 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:40.944905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:41.046630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d283-2a0d-42a1-be0b-49759d32758b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:41.084772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:41.085433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.085717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.085996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.089601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:41.165899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:41.167314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:41.272460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 163/316] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:41.277776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:41.370926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:41.380469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b53fc-c453-469f-8698-f1d270b68656 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:41.416961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:41.417802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.418087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.418363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.419584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:41.421011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21d1c-2880-4271-9389-289a37ca2279 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:41.428952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 154/317] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:41 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:41.433309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:41.468878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:41.542317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec81b-6289-4b8f-99e2-0e5b8210f1f6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:41.581169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:41.581871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.582152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.582460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.583295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, scope.project.id=92526b3d97d842718cfd684615dba767, user.id=372fdbb565c046898241245fd2fdcbf7) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:41.583605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:41.585038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6484d7-20fc-4979-8a92-5897b6220bf3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:41.591653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 164/318] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:01:41 2022] GET /identity/v3/role_assignments?user.id=372fdbb565c046898241245fd2fdcbf7&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:41.653374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288aba3-13b7-4016-b94c-170ad160f058 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.653751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288aba3-13b7-4016-b94c-170ad160f058 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.654097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288aba3-13b7-4016-b94c-170ad160f058 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.655842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 155/319] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:41.661924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.662347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.662721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.681472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:41.682675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:41.758929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 165/320] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:41.781818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288aba3-13b7-4016-b94c-170ad160f058 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:41.883619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288aba3-13b7-4016-b94c-170ad160f058 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:41.918675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:41.919473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:41.919737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:41.920008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:41.921185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:41.922713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fef2d-c1f1-409b-a347-9658a89bc873 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:41.924688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 156/321] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:41.930222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:41.960455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.018402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff463d-171b-4eb8-86cd-94af707edc29 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.018743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff463d-171b-4eb8-86cd-94af707edc29 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.018971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff463d-171b-4eb8-86cd-94af707edc29 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.020493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 157/322] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:42.025759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aece47-9548-463b-bf67-8fcff0d32aff None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.026051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aece47-9548-463b-bf67-8fcff0d32aff None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.026329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aece47-9548-463b-bf67-8fcff0d32aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.032259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6f118-b602-4360-8bbb-270be56478c1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.043728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-19aece47-9548-463b-bf67-8fcff0d32aff None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:42.044918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-19aece47-9548-463b-bf67-8fcff0d32aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:42.067822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.078579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.079252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.079584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.079858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] PATH_INFO: `/v3/groups` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.080678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.082212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.098102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 166/323] 10.222.0.16 () {60 vars in 1134 bytes} [Mon Jul 4 04:01:41 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:01:42.124997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 158/324] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:42.148331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.162994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78fa4c-4379-4126-b338-bb14f72e4651 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.163284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78fa4c-4379-4126-b338-bb14f72e4651 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.163561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78fa4c-4379-4126-b338-bb14f72e4651 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.165010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 159/325] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:42.171069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.171254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.171410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.189231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:42.190667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:42.251556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.264904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204370d9-7003-4807-b0b9-b74d0f45c1fc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.266409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 160/326] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:42.290940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.306058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.307154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.307432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.307706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.318602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:42.328319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:42.329270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:42.330156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 167/327] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:42.334798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.397141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.414245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f91ba-c8ef-4715-9aca-81ed04ee3810 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.451910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb264-500b-4427-a924-bf641ba5cb14 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.453761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.454548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.454827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.455097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.476668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:42.486500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:42.487434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:42.488218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:42.489518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 161/328] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:42.493749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.499936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.500135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.500358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.500569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.501889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:42.502226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.505273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.521333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 168/329] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:42.528156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.562815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.621970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360bf45-f7de-4aed-8a3c-d2d02b5f5eae None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.656677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323450b-34af-4790-90d6-5e6c713a7eb2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.670513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.671455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.671886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.672434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.673500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:42.674294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.675874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.685539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 162/330] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:42.695074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.695504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.696398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.696876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.697322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] PATH_INFO: `/v3/users/500182e9c0774682a5ebebd9f4290eef` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.698734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] RBAC: Authorizing `identity:get_user(user_id=500182e9c0774682a5ebebd9f4290eef)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.700502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.702424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 169/331] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/users/500182e9c0774682a5ebebd9f4290eef => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:42.712047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.713398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:42.811221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:42.834744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612cc54-4d5e-4786-ae39-3c94a61c2858 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.861575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d28ee-512e-4295-a31a-bffda5cc2580 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:42.877771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.878704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.879093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.879370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] PATH_INFO: `/v3/system/users/500182e9c0774682a5ebebd9f4290eef/roles/c024d0f9c15d42f080c58d0cd5fc6e77` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.880870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=500182e9c0774682a5ebebd9f4290eef, role_id=c024d0f9c15d42f080c58d0cd5fc6e77)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.882394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.897749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 170/332] 10.222.0.16 () {58 vars in 1334 bytes} [Mon Jul 4 04:01:42 2022] PUT /identity/v3/system/users/500182e9c0774682a5ebebd9f4290eef/roles/c024d0f9c15d42f080c58d0cd5fc6e77 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:42.911358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:42.912363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:42.912701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:42.912984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] PATH_INFO: `/v3/users/69815719db2248a7b1f5da5b4a7e9e92` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:42.915350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] RBAC: Authorizing `identity:get_user(user_id=69815719db2248a7b1f5da5b4a7e9e92)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:42.916928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa7327-6e11-4e2e-ab4d-f8eae4beb782 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:42.923586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 163/333] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/users/69815719db2248a7b1f5da5b4a7e9e92 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:42.930126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:43.077745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e134e-26e0-4fb5-8db4-bd3d7f37b575 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:43.118381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:43.119057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:43.119337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:43.119667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:43.120504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] RBAC: Adding query filter params (role.id=d4760ffd4761422abfabd45bc0880415, user.id=69815719db2248a7b1f5da5b4a7e9e92, scope.system=all) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:43.120866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:43.122349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f06a9e-9602-4555-bfb9-8dbd9aa6d135 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:43.135578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 171/334] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Jul 4 04:01:42 2022] GET /identity/v3/role_assignments?user.id=69815719db2248a7b1f5da5b4a7e9e92&scope.system=all&role.id=d4760ffd4761422abfabd45bc0880415 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:44.708186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d644-913b-4414-bd53-a394066c0aed None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:44.708681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d644-913b-4414-bd53-a394066c0aed None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:44.708791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d644-913b-4414-bd53-a394066c0aed None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:44.710362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 164/335] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:44.724188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50270e-193d-4592-817f-cdd63a73601c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:44.725103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50270e-193d-4592-817f-cdd63a73601c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:44.725577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50270e-193d-4592-817f-cdd63a73601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:44.731983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:44.747887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2b50270e-193d-4592-817f-cdd63a73601c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:44.749431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b50270e-193d-4592-817f-cdd63a73601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:44.840020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 172/336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:44.878572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555d644-913b-4414-bd53-a394066c0aed None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:44.979998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:45.009831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555d644-913b-4414-bd53-a394066c0aed None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:45.047465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:45.048368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.048652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.049122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.050669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22768-0853-4d98-9fee-0b0d24d74089 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.050953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22768-0853-4d98-9fee-0b0d24d74089 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.051228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22768-0853-4d98-9fee-0b0d24d74089 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.052741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 173/337] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:01:45.057350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.057635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.057912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.075256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:45.075556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:45.076664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:45.083995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:45.084846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:45.085621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:45.094394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 165/338] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:45.095763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:45.215985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 174/339] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:45.250095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:45.318107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13011d-d084-4823-b0a6-56b9255be44c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:45.377923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:45.407753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:45.409025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.409502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.410259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.411256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:45.411756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:45.413341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968212a-cd70-4399-8be2-b16a251b8dba None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:45.426404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:45.426809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 166/340] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:45.428440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c14394-f4e8-4c5e-9dde-da197864d40c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.428918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c14394-f4e8-4c5e-9dde-da197864d40c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.429384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c14394-f4e8-4c5e-9dde-da197864d40c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.431187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 167/341] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:45.433881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c14394-f4e8-4c5e-9dde-da197864d40c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:45.449721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fc498-4d62-4b93-b83d-00823328a21e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:45.515788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:45.516539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.517834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.517834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.539077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1a4a5a6f-1bf8-4a68-a19a-4a0fff9e11c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:45.548240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:45.549151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:45.549910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:45.552168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 175/342] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:45.553273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.553568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.553853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.573529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c14394-f4e8-4c5e-9dde-da197864d40c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:45.588411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:45.589611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:45.654223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:45.656235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:45.656522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:45.656815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:45.745428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:45.750869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:45.752583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:45.756388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 176/343] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:45.757269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:45.899649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:45.906214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 168/344] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:45.907204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.017794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953bc1-df34-4cfd-8473-445e33200e94 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.041077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.048009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb6a39-5dd9-4d38-8ce8-f435ebaa9ac9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.070161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.071268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.071746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.072209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.073293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:46.073918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.075530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.087245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 177/345] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:46.088495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.088786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.089715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.089999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.090268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.104210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-378c22f6-67fe-4398-a54e-56481a365b9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:46.112413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:46.113347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 169/346] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:46.118008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23027d0-dbe9-4db2-b76d-534abc60eaf3 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.118956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23027d0-dbe9-4db2-b76d-534abc60eaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.118956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23027d0-dbe9-4db2-b76d-534abc60eaf3 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.118956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 170/347] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:46.118956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23027d0-dbe9-4db2-b76d-534abc60eaf3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.222685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb34752-5761-430a-8d73-4f9d23ae0de7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.237231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.245713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23027d0-dbe9-4db2-b76d-534abc60eaf3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.260345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.261487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.261942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.262346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.263722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.265823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.268636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 178/348] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:46.269606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.286967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.287763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.288148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.288818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.319868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.366448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.367939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.393206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979359-616e-463e-affd-84ea6996cd0a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.436913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.437194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.437466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.438226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:46.438742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.440639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce20cd-5d91-4536-8db6-462afb465115 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.450362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 179/349] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:46.451599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d1bd-e1d1-4c16-82d9-474bd919b751 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.451878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d1bd-e1d1-4c16-82d9-474bd919b751 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.452154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d1bd-e1d1-4c16-82d9-474bd919b751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.469785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f3f6d1bd-e1d1-4c16-82d9-474bd919b751 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:46.470569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f3f6d1bd-e1d1-4c16-82d9-474bd919b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:46.494831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 171/350] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:46.495416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.520589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.560800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 180/351] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:46.562124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c717e-8b16-45b4-91da-bfcae3f8d2cc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.563117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c717e-8b16-45b4-91da-bfcae3f8d2cc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.563362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.563476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c717e-8b16-45b4-91da-bfcae3f8d2cc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.565603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 181/352] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:46.566305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c717e-8b16-45b4-91da-bfcae3f8d2cc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.609784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c002c-83e5-46be-a42e-119bc1c2ef9c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.660098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.661023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.661303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.661575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.668899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.670347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.672312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 172/353] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:46.672894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.703393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c717e-8b16-45b4-91da-bfcae3f8d2cc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.742353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.744126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.744342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.744617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.803723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a294-201e-4766-aee5-87bc196caeeb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:46.804646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.835939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:46.849033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:46.850187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.850507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.850790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.851266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.852110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:46.853158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.853597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852fc53-965a-40ae-97f3-0fb55268ef56 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:46.855618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 173/354] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:46.858709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:46.859000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:46.859276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:46.877920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:46.879876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:46.968331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 174/355] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:46.969177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:46.981697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 182/356] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:46.982389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.064936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.087846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bee1f-5392-4713-81aa-5500d7995ba0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.096765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e502f3-0c3e-4a0c-8c4b-0414b7851783 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.118446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.132634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.133746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.134026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.134299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.138424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.139091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.139375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.139647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.140480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:47.140794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.142601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.148136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:47.148338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 183/357] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=d4760ffd4761422abfabd45bc0880415 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.151258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d7e44f73-db41-4781-81fa-c13926fcd54d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:47.156823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:47.157900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:47.159215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 175/358] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:47.160103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f637c-a10c-4c5c-9d14-104a8eb20fc1 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.161388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f637c-a10c-4c5c-9d14-104a8eb20fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.161388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f637c-a10c-4c5c-9d14-104a8eb20fc1 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.162148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 176/359] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:47.164602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f637c-a10c-4c5c-9d14-104a8eb20fc1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.289590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12ad7-7f1e-4552-83a9-b9671d2bc8d7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.311333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f637c-a10c-4c5c-9d14-104a8eb20fc1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.332421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.333222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.333504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.333779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.336322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.337805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.339740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 184/360] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.340508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.350148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.350457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.351311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.351591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.351864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.353055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.354579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.356422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 177/361] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.357181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.432264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.469440 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d35f-2d58-4e0f-9cc1-8314b7fa8f94 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.495964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be70801-3d3d-4570-921a-5b12c99041e5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.515212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.516021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.516305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.516574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.529268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d9a7c16c-f1e1-48c2-ba5f-9f62ca94c566 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:47.537992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:47.539192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:47.540145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 185/362] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:47.541125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.541409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.541687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.544608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.545553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.545989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.546439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.547423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:47.547796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.549343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.559018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 178/363] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.559018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.560776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:47.562228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:47.638893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.649726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 186/364] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:47.650953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.686496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e45eb-7fb2-4201-9307-e52736ad876a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.700871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.732166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.732921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.733103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.733376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.734214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=92526b3d97d842718cfd684615dba767) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:47.734639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.736185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.749926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 179/365] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=d4760ffd4761422abfabd45bc0880415 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.752491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.792623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f742ce-5e43-415b-9d54-ea4d063f3fe4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.837419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.838351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.838666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.838942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.840209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.841660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.843332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 187/366] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.844095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:47.907495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d7f92-0437-4662-aa7c-8ea44bdc3741 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.920340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:47.957860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:47.958858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:47.959143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:47.959728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:47.960954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:47.961587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:47.963058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:47.963337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1a081-e73e-400b-a9bc-de98549cc73c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:47.980498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 180/367] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:47.981349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.010982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.011783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.012075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.012354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.033768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.096845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.098299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.115959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aea8b-fdb0-4d7d-9724-bdd5e8687850 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.161850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.162649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.162936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.163213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.169533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.178752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:48.187631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:48.188551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:48.189398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 181/368] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:48.190067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.219868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 188/369] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:48.221297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.308565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.316531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b08c-a889-47fa-864e-769028fc76f1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.346862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9032161-0104-4e6f-ab14-e8be4c3917a4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.357802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.358011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.358295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.358711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.359633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:48.360014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.361782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.368218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 182/370] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:48.369819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.395167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.396877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.397393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.397874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] PATH_INFO: `/v3/users/500182e9c0774682a5ebebd9f4290eef` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.399487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] RBAC: Authorizing `identity:get_user(user_id=500182e9c0774682a5ebebd9f4290eef)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.402162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.405200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/users/500182e9c0774682a5ebebd9f4290eef => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:48.410767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.504386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a30de2-8c2f-499e-94ac-d1ab6bdd07a9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.509318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.550715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.551823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.552308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.552784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.553830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:48.554334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.556052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.568706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 183/372] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:48.570099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.598536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.617510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32932-d17e-4bd6-a36c-08c3aee65958 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.662251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.663542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.664030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.664504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.665942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.667668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.669954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:48.671026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.688182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc41f6-2735-4936-87fc-4d0fac8b39d0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.736450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.737128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.737413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.737689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.742658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] RBAC: Adding query filter params (role.id=c024d0f9c15d42f080c58d0cd5fc6e77, user.id=500182e9c0774682a5ebebd9f4290eef, scope.system=all) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:48.743176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.745014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.761513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1308 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/role_assignments?user.id=500182e9c0774682a5ebebd9f4290eef&scope.system=all&role.id=c024d0f9c15d42f080c58d0cd5fc6e77 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:48.762663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:48.774784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.777259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09f63b-38bb-4a1b-9433-1a7fc423efcb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.816038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.816689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.817118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.817449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.891010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.892789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.916035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:48.944648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443837-7b43-4036-bdf2-ebf8fd3298e1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:48.988098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:48.989467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:48.990055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:48.990554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:48.992030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:48.993810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:48.998128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 185/375] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:49.003483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:49.005248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:49.008548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 191/376] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:49.014125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:49.175019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f106-272d-4fdb-b5a0-776fc0536660 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:49.211015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6cad0-78fd-4ecc-9c42-f64a04a387e0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:49.248256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:49.249186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.249477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.250166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.251430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=92526b3d97d842718cfd684615dba767) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:49.251720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:49.253377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:49.255527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:49.256410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.256695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.256972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] PATH_INFO: `/v3/projects/525d4bbd65cc40f382674c9a7977e83e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.258170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] RBAC: Authorizing `identity:get_project(project_id=525d4bbd65cc40f382674c9a7977e83e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:49.259970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:49.263566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 186/377] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:49 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:49.266837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 192/378] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:49 2022] GET /identity/v3/projects/525d4bbd65cc40f382674c9a7977e83e => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:49.267288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:49.299055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:49.420041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c542-24bc-4306-954d-b152e643e3ce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:49.462018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:49.462945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.463233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.463513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] PATH_INFO: `/v3/groups/2468800fcaf3402082e7ad27119b4077` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.471809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] RBAC: Authorizing `identity:get_group(group_id=2468800fcaf3402082e7ad27119b4077)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:49.473301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f6f84-525a-4998-8c50-63eaa7f2d8f2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:49.475442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 187/379] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:49 2022] GET /identity/v3/groups/2468800fcaf3402082e7ad27119b4077 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:49.480899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:49.570453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:49.612243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e224d-30d9-4fdb-a982-478b2bc9f1ff None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:49.651410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:49.652075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.652357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.652634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.653476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] RBAC: Adding query filter params (group.id=2468800fcaf3402082e7ad27119b4077, role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=525d4bbd65cc40f382674c9a7977e83e) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:49.653789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:49.655261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63595966-5350-43f5-895c-c5770e8f9eb2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:49.662091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 193/380] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:49 2022] GET /identity/v3/role_assignments?group.id=2468800fcaf3402082e7ad27119b4077&scope.project.id=525d4bbd65cc40f382674c9a7977e83e&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:49.774963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37b77-f101-4732-9d92-f32fb1aad2d2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.775297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37b77-f101-4732-9d92-f32fb1aad2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.775841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37b77-f101-4732-9d92-f32fb1aad2d2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.777774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 188/381] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:49.783991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:49.784577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:49.785073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:49.804013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:49.805595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:49.883366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 194/382] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:49.904024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f37b77-f101-4732-9d92-f32fb1aad2d2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:49.998752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.022594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f37b77-f101-4732-9d92-f32fb1aad2d2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.059136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791918-8b3d-434e-878e-e60118826171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.060013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791918-8b3d-434e-878e-e60118826171 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.060339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791918-8b3d-434e-878e-e60118826171 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.060628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791918-8b3d-434e-878e-e60118826171 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.073068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791918-8b3d-434e-878e-e60118826171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-10791918-8b3d-434e-878e-e60118826171 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:50.082027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:50.082964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:50.083744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:50.088415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 189/383] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:50.089804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.156523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.197638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f01ecf-d5c3-45a3-b52b-e51ea8283c8e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.234386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.235081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.235371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.235707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.236477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:50.236790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:50.238176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0184f7-893e-45bc-851a-65d493268c9e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:50.247307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 195/384] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:50.253191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791918-8b3d-434e-878e-e60118826171 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.276303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a14a9-0759-420b-b177-5d15c15154ad None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.276588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a14a9-0759-420b-b177-5d15c15154ad None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.276870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a14a9-0759-420b-b177-5d15c15154ad None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.278257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 196/385] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:01:50.283732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.284023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.284316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.305784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:50.307014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:50.397576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 197/386] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:50.404233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791918-8b3d-434e-878e-e60118826171 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.422778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.449564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.450543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.450826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.451101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.455289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:50.458892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:50.458892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 190/387] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:50.466484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.479269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.554248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1db4e-8b0c-4837-8368-ce0b34b144a0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.589426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.590238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.590562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.590839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.601960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:50.614392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:50.615314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:50.616095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 198/388] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:50.620211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.667823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4829bf-90c8-4424-a07b-337f5ad52585 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.750501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1da2c2-6d75-46dd-9a1e-492cc069fa2d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.768165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.775669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.776568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.776851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.777127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.783168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:50.784713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:50.786699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 191/389] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:50.787547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:50.788390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:50.788819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:50.789242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:50.790165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:50.790647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:50.792121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.792705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:50.802705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 199/390] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:50.811367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:50.913699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:50.963558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf573ac-7e09-4fa6-9e39-024bf7aa5620 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:50.981632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c3523-64da-4aac-b58f-ef694a6fb999 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:51.003721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.005187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.005611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.006041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/groups/059ecff880474a96bbab04344c00e68d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.008478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, group_id=059ecff880474a96bbab04344c00e68d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:51.010914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:51.027301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 192/391] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Jul 4 04:01:50 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/groups/059ecff880474a96bbab04344c00e68d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:01:51.061258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.062149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.062450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.062726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.067296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:51.068776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:51.070923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 200/392] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:50 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:51.076455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:51.210036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b145d-e7f0-4c84-940e-c2e0ef70fa5f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:51.247040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.247956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.248243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.248526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.249980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:51.251789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c1c5-479a-4d8c-9887-a4dd310ffa76 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:51.253958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 193/393] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:51 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:51.263437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:51.270317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b76b-11df-4a12-89c6-bcafa1277e08 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.270630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b76b-11df-4a12-89c6-bcafa1277e08 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.270966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b76b-11df-4a12-89c6-bcafa1277e08 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.272537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 194/394] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:51.277596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.277881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.278159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.285466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:51.302300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:51.303543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:51.338107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:51.420935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d53579-3e7d-44dc-971f-34651f6f0c24 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:51.425311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 195/395] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:51.449334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:51.486544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.487877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.488362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.488835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/groups/059ecff880474a96bbab04344c00e68d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.491124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, group_id=059ecff880474a96bbab04344c00e68d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:51.492845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:51.523057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 201/396] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Jul 4 04:01:51 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/groups/059ecff880474a96bbab04344c00e68d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:51.569919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e3b8-8c6c-48ff-bfb2-ff3ac5da94f7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:51.608531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:51.614734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.615645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.615931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.616206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.643820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:51.652426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:51.653384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:51.654697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 196/397] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:51.662433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:51.820141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ec790-c023-409f-b0a5-627f2054b6d3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:51.822949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:51.861374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:51.862473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:51.863022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:51.868607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:51.869814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:51.870330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:51.872072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:51.886255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 202/398] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:51.892455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.007419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813dbf0-ac1e-4baa-b247-e51c88eeb02f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.053186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.054510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.055021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.055496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.056947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:52.058631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c994a-9248-40a9-a843-8883e90ca999 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:52.065227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 197/399] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:51 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:52.067499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.096572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d828-3b1f-44be-a1d8-08773e3e124f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.097169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d828-3b1f-44be-a1d8-08773e3e124f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.097647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d828-3b1f-44be-a1d8-08773e3e124f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.099718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 198/400] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:52.105504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.106051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.106590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.130023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:52.133240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:52.206529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:52.223712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 199/401] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:52.256494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eb4d4-c4aa-48b3-87f0-d9a913291793 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.257844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.333864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.335703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.336278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.336786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.338280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:52.340110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2de3e-c6ed-4291-8a5c-4b4738345381 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:52.346547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 203/402] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:52.348219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b400d-4021-4d44-bcf5-723c1417f21f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.348797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b400d-4021-4d44-bcf5-723c1417f21f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.350899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b400d-4021-4d44-bcf5-723c1417f21f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.352772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 204/403] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:52.354494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b400d-4021-4d44-bcf5-723c1417f21f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.394908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345776b8-d2f9-43c5-9c59-411ce51b74e3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.397302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:52.429809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.430657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.430940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.431214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.441759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c408841a-8760-4589-b174-cd8b921de3ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:52.450231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:52.451395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:52.452506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 200/404] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:52.453257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.453257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.453257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.475083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:52.476272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:52.482773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b400d-4021-4d44-bcf5-723c1417f21f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.524239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:52.525519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.527496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.527976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.528593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/groups/059ecff880474a96bbab04344c00e68d/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.531251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b52deef3bbe43eeb8063fe7f16f7af8, group_id=059ecff880474a96bbab04344c00e68d, role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:52.533781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:52.549421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 201/405] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:52.557543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.558342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 205/406] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Jul 4 04:01:52 2022] PUT /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8/groups/059ecff880474a96bbab04344c00e68d/roles/899a8afb374942c8872cc2929f6b4b1a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:52.598194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.731671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:52.744305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea401b12-1b42-4a7d-913f-1aa108620f19 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.780808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a2927-0775-48c3-bb6d-0fcdf9aebc48 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.789139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.790186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.790658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.791079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.805414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:52.816178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:52.817350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:52.818338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:52.821291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 206/407] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:52.831210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.853718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.855065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.855496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.855917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.856823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:52.857292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:52.859709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:52.872675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 202/408] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:52.881529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:52.943052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:52.951267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca105-b586-4dc5-97ec-b1a57e6acaf6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:52.986877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114898a-9aa2-4697-907d-88769696617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:52.987578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114898a-9aa2-4697-907d-88769696617f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:52.987858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114898a-9aa2-4697-907d-88769696617f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:52.988129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114898a-9aa2-4697-907d-88769696617f None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:52.988888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114898a-9aa2-4697-907d-88769696617f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:52.989191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114898a-9aa2-4697-907d-88769696617f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:52.990639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114898a-9aa2-4697-907d-88769696617f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:52.999884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 207/409] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:53.011087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114898a-9aa2-4697-907d-88769696617f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.031857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:53.047715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c7ead-caf9-42e5-8ade-96de9aa4abfa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.107450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.108719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.109191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.109673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.111119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.112958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.120419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 203/410] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:52 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:53.127996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.158507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114898a-9aa2-4697-907d-88769696617f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.201068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.201862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.202143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.202439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] PATH_INFO: `/v3/groups/2468800fcaf3402082e7ad27119b4077` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.204419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] RBAC: Authorizing `identity:get_group(group_id=2468800fcaf3402082e7ad27119b4077)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.205934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.215290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 208/411] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:53 2022] GET /identity/v3/groups/2468800fcaf3402082e7ad27119b4077 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:53.215921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.342164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25220b-ee46-407d-834d-dc33fd266853 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.347069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:53.361970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:53.365231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b118982-087e-43ee-b0f5-c7bb42cdc561 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.401372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.402250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.402563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.402842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] PATH_INFO: `/v3/projects/525d4bbd65cc40f382674c9a7977e83e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.404104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] RBAC: Authorizing `identity:get_project(project_id=525d4bbd65cc40f382674c9a7977e83e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.405613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.408201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 209/412] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:53 2022] GET /identity/v3/projects/525d4bbd65cc40f382674c9a7977e83e => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:53.413954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.443679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.444574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.444855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.445131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.451166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.452578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.454736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 204/413] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:53 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:53.461858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.525212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2a8a0-0cef-4f46-b3e9-df716ac6db92 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.561151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.562049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.562334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.562642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] PATH_INFO: `/v3/projects/525d4bbd65cc40f382674c9a7977e83e/groups/2468800fcaf3402082e7ad27119b4077/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.564792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=525d4bbd65cc40f382674c9a7977e83e, group_id=2468800fcaf3402082e7ad27119b4077, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.566249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a146d4d-2eb7-4011-ae79-83e20fc01a5c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.584799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 210/414] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Jul 4 04:01:53 2022] PUT /identity/v3/projects/525d4bbd65cc40f382674c9a7977e83e/groups/2468800fcaf3402082e7ad27119b4077/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:01:53.586225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425056-7132-49fe-acb6-9d4bd657b327 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.590631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425056-7132-49fe-acb6-9d4bd657b327 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.590959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425056-7132-49fe-acb6-9d4bd657b327 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.592864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 211/415] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:53.599964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.600302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.600523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.636879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:53.642505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:53.671385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:53.698107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8916-935b-447e-a306-3e48d31e98be None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.769138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 212/416] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:53.779946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.781213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.781213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.781386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767/groups/059ecff880474a96bbab04344c00e68d/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.783829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=92526b3d97d842718cfd684615dba767, group_id=059ecff880474a96bbab04344c00e68d, role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:53.785268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf14830-e93d-42f9-a888-d4c03e4c45c3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:53.801846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 205/417] 10.222.0.16 () {58 vars in 1442 bytes} [Mon Jul 4 04:01:53 2022] PUT /identity/v3/projects/92526b3d97d842718cfd684615dba767/groups/059ecff880474a96bbab04344c00e68d/roles/899a8afb374942c8872cc2929f6b4b1a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:01:53.824753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:53.936216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6f81-a547-48a0-aed5-3c66f3f3f8ac None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:53.975865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:53.976698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.976976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.977246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.996007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd63e0-6f53-49f3-8064-cba5ed1a12b3 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:53.996333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd63e0-6f53-49f3-8064-cba5ed1a12b3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:53.996564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd63e0-6f53-49f3-8064-cba5ed1a12b3 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:53.998007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 206/418] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:54.003108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:54.004767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.005255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.005780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:54.010008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:54.010902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:54.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 213/419] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:54.023497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.035288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.035969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:54.037614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:54.074996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.123346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 207/420] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:54.146056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24ea9-4f2a-4ac5-b569-56596bc82d61 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.147400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.229486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.230602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.231084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.231583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.232532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] RBAC: Adding query filter params (name=member) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:54.233033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:54.234663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:54.255177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 214/421] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:54.259923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.280144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33af138-c0df-4d6f-a31c-d009cdb0eb5a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.317397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.318612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.319118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.319589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.348970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:54.359949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:54.361446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 04:01:54.362230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:54.368595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 208/422] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:01:54.374576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.378705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.410479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.430485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbd754-9380-41a8-a417-2e3c7e745540 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.482390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.483303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.483596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.483878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.552803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecec8b-48f5-4276-99fb-f20bc7c70592 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.566093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:54.568156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:54.637815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.638643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.638928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.639205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.639978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:54.640284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:54.641811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:54.652364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.672323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 209/423] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:54.679503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.697362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 215/424] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 04:01:54.702886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:54.854476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:54.861778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53cf2-0ac8-4297-8481-5b3dc75cab91 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.872582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115f4e-256c-4a70-b10c-703dfeb728f4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:54.928537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.929200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.929480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.929754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.939498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:54.940383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:54.940711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:54.940995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:54.947402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:54.948866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:54.950923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 216/425] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:54.975290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.036317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.077367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.078618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbddb9d-78c2-42bf-bab0-8ea61a89b0cb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.136932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5629-1646-4816-8052-a0549eb7846e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.174925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.178912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 210/426] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:55.180325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b3d2-b658-42df-8181-ea8a2196c24c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.180701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b3d2-b658-42df-8181-ea8a2196c24c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.181058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.181186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b3d2-b658-42df-8181-ea8a2196c24c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.182162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.182612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.186456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 211/427] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:55.186588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.187828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8b3d2-b658-42df-8181-ea8a2196c24c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.188855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.190303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beecb95-4f6a-4c5c-817a-e69a6a416699 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.192563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 217/428] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:54 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:55.194570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.194570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.194570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.215630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:55.216879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:55.294441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 218/429] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:55.294441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.336331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.337135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8b3d2-b658-42df-8181-ea8a2196c24c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.376703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.377854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.378317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.378837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.380265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.382007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.384433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 212/430] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:55.385704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.426864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855237f-9217-49c8-b9f4-15902efa0f95 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.448619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.474920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.475749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.476126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.476411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.477253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:55.477568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.479707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.487596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 219/431] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=d4760ffd4761422abfabd45bc0880415 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:55.488641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.495173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf76168-68ac-47ae-b1cf-05ebaa20b3f0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.538802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.540075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.540586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.541048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.553095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:55.562175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:55.563132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:55.564229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:01:55.566918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 213/432] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:55.580603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.619803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbf9f4-9e41-4562-96c6-ce741987f058 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.626648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.718979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.720480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.721119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.721585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.723055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.724686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.726818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 220/433] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:55.733924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.755486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d72d7d-c3b7-499b-8089-739e2f30bed9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.782316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:55.805031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.805869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.806155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.806772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.807826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:55.808143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.810174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.823040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 214/434] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:55.830978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:55.867649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380d846-3e62-4d95-8dbf-987d881520c9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:55.919030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:55.919699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:55.919981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:55.920256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:55.921250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=d4760ffd4761422abfabd45bc0880415, scope.project.id=92526b3d97d842718cfd684615dba767) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:55.921562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:55.924152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30aa3b-4b07-4c10-a739-da8b4bcb0737 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:55.938549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 221/435] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=d4760ffd4761422abfabd45bc0880415 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:55.977927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7974249-f3fb-4505-991b-66500af25e60 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.023184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268b966-ced9-4618-bf17-06da46b3d674 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.023500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268b966-ced9-4618-bf17-06da46b3d674 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.023754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268b966-ced9-4618-bf17-06da46b3d674 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.025322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 222/436] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:56.027018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.027950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.028440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.028948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.030814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.031102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.031423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.055362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:56.056684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:56.082562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.104492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.131491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.133522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.139981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 223/437] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:01:56.165822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.251415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 215/438] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 04:01:56.257030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.313362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe77c6e-e2d9-41a4-aba7-5ab880d8e144 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.350083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.350910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.351188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.351453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.354831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.361703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b511be8a-20b4-4af3-a0b1-a31dbb0a72b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:56.369449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:56.370348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:56.371165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 224/439] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:56.372023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b608c-df4b-47f0-b5fb-21d6b0eba826 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.372191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b608c-df4b-47f0-b5fb-21d6b0eba826 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.372456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b608c-df4b-47f0-b5fb-21d6b0eba826 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.373842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 225/440] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:01:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:01:56.376601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b608c-df4b-47f0-b5fb-21d6b0eba826 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.383665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d6aa4-35e1-4dac-b3e5-f4ea47a49e7d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.397041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.417930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.418827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.419107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.419380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] PATH_INFO: `/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.420633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] RBAC: Authorizing `identity:get_project(project_id=8b52deef3bbe43eeb8063fe7f16f7af8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.422066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef050-b766-42dd-b372-9deaf131fa86 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.423821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 216/441] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/projects/8b52deef3bbe43eeb8063fe7f16f7af8 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:56.425009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.425293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.425565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.441509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:01:56.442713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:01:56.487873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b608c-df4b-47f0-b5fb-21d6b0eba826 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.512592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 217/442] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:01:56.513325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.532429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.533434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.533894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.534882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.536116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:56.536581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.538511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.550509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 226/443] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:56.550509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.597517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.619476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5e2bd-9b4c-488b-ba55-80629775aa6a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.630476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.653262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7753d3-b9c5-4a90-98a5-6d664f4ce95b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.660821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.661711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.661991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.662265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.663484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.665166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.666810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 218/444] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:56.667592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.695238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.696062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.696349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.696628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.707472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:01:56.715625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:01:56.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:01:56.717519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 227/445] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:01:56.718585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.771831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e60b-4ea1-41bc-988c-05375cce317e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.806601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.807257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.807535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.807810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.818079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd8c7-678f-4b5f-a459-f61a82ec7e62 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:56.823785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.851093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:56.851740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:56.852019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:56.852292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:56.853115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:56.853428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.854886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.862831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 228/446] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=8b52deef3bbe43eeb8063fe7f16f7af8&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:56.863608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:56.866071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:56.890078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:56.891707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:56.977130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3254-59c6-4485-8b3a-81343020f699 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.011386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 219/447] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:01:57.012227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.014437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.015117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.015402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.015679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.016437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:57.016872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.018256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.027393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 229/448] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:01:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:57.032569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.063230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.113603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.129947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa3eec-bd66-464e-b61d-e61083303d32 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.159063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e761e-81a5-4c25-ab83-2a1fc4644155 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.173994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.174903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.175185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.175458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] PATH_INFO: `/v3/projects/525d4bbd65cc40f382674c9a7977e83e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.176657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] RBAC: Authorizing `identity:get_project(project_id=525d4bbd65cc40f382674c9a7977e83e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.178097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.180198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 220/449] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/projects/525d4bbd65cc40f382674c9a7977e83e => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:57.185504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.201500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.202161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.202473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.202747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.290418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.291809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.311781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984aa38-52cb-4605-8449-c7da17205e10 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.328198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.356795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.357665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.357944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.358216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] PATH_INFO: `/v3/groups/2468800fcaf3402082e7ad27119b4077` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.358973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.361279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] RBAC: Authorizing `identity:get_group(group_id=2468800fcaf3402082e7ad27119b4077)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.361635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.364420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 221/450] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/groups/2468800fcaf3402082e7ad27119b4077 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:57.370402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.403527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 230/451] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 04:01:57.409110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.474942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566afd-9545-405e-939c-d7ecb1b7b183 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.502552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293be684-2d3d-4ac7-987e-6f7bd15afe3a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.508371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.509025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.509304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.509887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.510740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] RBAC: Adding query filter params (group.id=2468800fcaf3402082e7ad27119b4077, role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=525d4bbd65cc40f382674c9a7977e83e) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:57.511051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.512464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.518308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 222/452] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/role_assignments?group.id=2468800fcaf3402082e7ad27119b4077&scope.project.id=525d4bbd65cc40f382674c9a7977e83e&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:57.536861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.537754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.538034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.538307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] PATH_INFO: `/v3/projects/92526b3d97d842718cfd684615dba767` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.539569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=92526b3d97d842718cfd684615dba767)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.541138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.542886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 231/453] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/projects/92526b3d97d842718cfd684615dba767 => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:01:57.549415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.603430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.663727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e3f6-1a29-45b5-bbd1-a7c3f3df0156 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.700147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.700982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.701260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.701533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] PATH_INFO: `/v3/groups/059ecff880474a96bbab04344c00e68d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.702740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] RBAC: Authorizing `identity:get_group(group_id=059ecff880474a96bbab04344c00e68d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.704309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4df135-289a-4a4c-9fb4-1e711dc5a7a2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.705940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 223/454] 10.222.0.16 () {56 vars in 1180 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/groups/059ecff880474a96bbab04344c00e68d => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:01:57.712111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:01:57.743315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:01:57.812713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7623c7c-3255-4eec-8862-63fd44dd6fc0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:01:57.850030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:01:57.850701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:01:57.850979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:01:57.851248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:01:57.852067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] RBAC: Adding query filter params (group.id=059ecff880474a96bbab04344c00e68d, role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=92526b3d97d842718cfd684615dba767) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:01:57.852381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:01:57.853795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa5c1-c6c8-49a5-9876-4c2d9506d669 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:01:57.859764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 232/455] 10.222.0.16 () {56 vars in 1376 bytes} [Mon Jul 4 04:01:57 2022] GET /identity/v3/role_assignments?group.id=059ecff880474a96bbab04344c00e68d&scope.project.id=92526b3d97d842718cfd684615dba767&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:00.550742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db97a-f7f9-461d-92f9-b1c92a3ce13a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.551269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db97a-f7f9-461d-92f9-b1c92a3ce13a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.551372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db97a-f7f9-461d-92f9-b1c92a3ce13a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.553053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 224/456] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:00.559061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.559917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.560383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.587366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:00.588900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:00.628751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b73e5-1eb6-43c4-8370-4a41e7928c3c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.629042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b73e5-1eb6-43c4-8370-4a41e7928c3c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.629486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b73e5-1eb6-43c4-8370-4a41e7928c3c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.631304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 225/457] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:00.636193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.637518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.637518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.667364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:00.668910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:00.697521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 233/458] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:00.726332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:00.778639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:00.807147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 226/459] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:00.830856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:00.879338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:00.884775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab1ad-0225-439b-a4ae-b09d4e0037ed None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:00.920491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:00.921554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.922001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.922458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.934476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:00.936258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:00.953870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618abc77-5518-4d89-a8b6-beba1b97625f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-cb17587a-445c-4e4d-b397-ac4f7ea522b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:00.955029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:00.955887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:00.956708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:00.956708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:00.956708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:00.956708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:00.958954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 234/460] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:00.960362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70237-72a8-49fe-b8d5-4018e7f115bb None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.960816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70237-72a8-49fe-b8d5-4018e7f115bb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.961255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70237-72a8-49fe-b8d5-4018e7f115bb None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:00.962708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 235/461] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:00.964563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70237-72a8-49fe-b8d5-4018e7f115bb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:00.993548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:00.994848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:00.995324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:00.995810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.006864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.008330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9c77f076-cc28-441e-b402-4ef10fbd5a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:01.029288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:01.030184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:01.031032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:01.031032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:01.031032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.031032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:01.033832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 227/462] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:01.035350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.035881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.036347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.056767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:01.058358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:01.065090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.090317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70237-72a8-49fe-b8d5-4018e7f115bb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.125656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.126732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.127238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.127699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.128609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:01.129100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.130718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db175eee-5768-4c56-8901-eeb0425a8817 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.138361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 228/463] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:01.139525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.147458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 236/464] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:01.147458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2595f-5b59-469e-99ff-9cee7b3baa27 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.147458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2595f-5b59-469e-99ff-9cee7b3baa27 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.147458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2595f-5b59-469e-99ff-9cee7b3baa27 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.147458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 237/465] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:01.151220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2595f-5b59-469e-99ff-9cee7b3baa27 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.173745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.249790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e9b85-2914-40a6-a87a-499de3dcf69e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.263769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2595f-5b59-469e-99ff-9cee7b3baa27 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.286044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.286790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.287071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.287347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.288071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:01.288374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.289746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c3c7d-d389-4136-9ffa-511ca03f6377 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.300048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.300730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.301016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.301293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.305884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 229/466] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:01.306009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.306106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.306106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.306106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.306280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.314210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.322885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:01.324068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:01.327672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 238/467] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:01 2022] POST /identity/v3/users => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:01.328424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.442988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 230/468] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:01.442988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.477710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.487928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f869389-ce00-4ede-bffa-0d4dd38e7fda None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.531496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.531815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.534189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.534189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.547186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.549231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:01.569688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:01.571151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:01.571991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:01.571991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:01.571991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.571991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:01.571991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 239/469] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:01.572559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.580907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aef4d-d131-486c-8851-c5fbadfd31c2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.629655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.632265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.632407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.633302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.633700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.635670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.639435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.661986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 231/470] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:01 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:01.662901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.709279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e0f1-d77d-4f8f-a64d-8d247c4d8f9f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.752921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.754552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.755056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.755585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.763420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.770915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.774126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:01.797533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:01.799162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:01.800068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:01.800068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:01.800068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:01.800068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:01.800814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccf4d-45ff-4761-b559-dae857f9bb2e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:01.802857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 240/471] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:01.814616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.846486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:01.847103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:01.847386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:01.847663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:01.851884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:01.851884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:01.851884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:01.866176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 232/472] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:01.871865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:01.980332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:01.983243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158a73-7fc2-4054-9f96-0d1f68bd4a52 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.008304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe580a-cb63-4457-acf6-e593af803502 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.019661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.020410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.020698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.020981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.021714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:02.022023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:02.023455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18fde8-1c01-4bcc-a985-5cca30c070c1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:02.037264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 241/473] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:02.039038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71435b-e013-4e56-9d52-48338e0df1e1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.039333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71435b-e013-4e56-9d52-48338e0df1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.039611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71435b-e013-4e56-9d52-48338e0df1e1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.044594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 242/474] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:02.044594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71435b-e013-4e56-9d52-48338e0df1e1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:02.051434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:02.058189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.059106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.059580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.060047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.060970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:02.062648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9059c4-ef94-433b-9697-85b481e86e7d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:02.087387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 233/475] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:01 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:02.089070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.089621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.090128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.109431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:02.111222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:02.168565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71435b-e013-4e56-9d52-48338e0df1e1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.194843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 234/476] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:02.219416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:02.234050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.234785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.235070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.235409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.236809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:02.238659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:02.273503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 243/477] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:02 2022] POST /identity/v3/users => generated 249 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:02.333048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3bfb-9035-4165-9825-daea5c04cbc6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.353623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:02.372606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.373214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.373496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.373774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.374642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:02.376810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:02.384943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.sql.core [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 04:02:02.384943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 04:02:02.384943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [parameters: {'id': '62379d761c774e93906d87a26a1668b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 04:02:02.384943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80370) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:02.402567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:02.403461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application yield resource Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise self.value Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application session.commit() Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 04:02:02.404332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.session.flush() Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application compat.raise_( Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 04:02:02.405166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:02.406021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:02.406873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [parameters: {'id': '62379d761c774e93906d87a26a1668b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:02.407691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 04:02:02.408615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:02.411837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 235/478] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Jul 4 04:02:02 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:02.413078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:02.432794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:02.525749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bda6f2-92b0-4a1e-b4b0-25debad765bf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.561528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.562353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.562654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.562926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.573696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-202a4594-8fd4-4838-b45f-0b93be21ec13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:02.582028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:02.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:02.583721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 244/479] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:02.588911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:02.700785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685dfa0-b6c1-4f21-a27f-34324ab05dd4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:02.739124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:02.739801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:02.740081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:02.740357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:02.741117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:02.741421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:02.742846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c31d82-5524-4d00-9c03-ddbba80a3c0b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:02.752345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 236/480] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:03.827394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b51b-f1bc-4db1-9f5f-c12c46c8e48d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:03.827926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b51b-f1bc-4db1-9f5f-c12c46c8e48d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:03.827926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b51b-f1bc-4db1-9f5f-c12c46c8e48d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:03.829817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 245/481] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:03.851047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:03.851543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:03.851639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:03.873378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:03.874577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:03.922810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:03.959689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 237/482] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:03.999729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b51b-f1bc-4db1-9f5f-c12c46c8e48d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.089132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:04.128074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b51b-f1bc-4db1-9f5f-c12c46c8e48d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:04.176321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:04.177051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.177332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.177603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.196241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-430f18f2-9d57-44f2-b9cb-51c4be16d712 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:04.203989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:04.204865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:04.205655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 246/483] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:04.210925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.274407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8adb6b-608a-44f2-8155-0dafd3796ab4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.274727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8adb6b-608a-44f2-8155-0dafd3796ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.275005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8adb6b-608a-44f2-8155-0dafd3796ab4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.276745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 247/484] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:04.282064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.282553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.283201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.300978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:04.302183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:04.349330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03a30-55e8-4cd5-8165-be477b684925 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:04.389122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:04.389803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.390087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.390388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.391149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:04.391566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:04.393557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:04.402645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 238/485] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:04.411673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.447341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 248/486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:04.464668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:04.519043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.551081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aa896-19a4-496f-9fc6-54d8e586425c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:04.606893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:04.649295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:04.650108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.650397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.650739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.663805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165d104-fff6-4065-927c-9ab5620f09fa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:04.674932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:04.676383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-21a722f9-7d59-4600-a152-9d1aff1ee829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:04.700670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:04.701549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:04.702397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:04.702397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:04.702397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:04.702397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:04.706730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 239/487] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:04.708296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c418-3948-4993-9314-ad8d74b48b7d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.708554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c418-3948-4993-9314-ad8d74b48b7d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.708830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c418-3948-4993-9314-ad8d74b48b7d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.710674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 240/488] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:04.712130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c418-3948-4993-9314-ad8d74b48b7d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.723955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:04.725228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.725587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.725937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.740240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2fe7a893-347e-4c8e-8a37-ff9319276042 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:04.750279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:04.752295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:04.754126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:04.762390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 249/489] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:04.762390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.762390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.762390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.783886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:04.785584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:04.821222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:04.837721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c418-3948-4993-9314-ad8d74b48b7d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:04.879902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 250/490] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:04.882400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:04.883086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:04.883405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:04.886389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:04.886389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:04.886389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:04.886389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:04.887339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.899809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 241/491] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:04.905611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:04.977135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.037466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceb5ef-870d-420b-866a-b4a45bc7d8dd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.039154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9bf80-45e6-4367-8afb-ab457aabe9f1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.084136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.084890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.085174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.085451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.091885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.092562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.092848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.093122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.093889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:05.094198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.095807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.096572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.097532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.112250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 251/492] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:05.113154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:05.116423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:05.117673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:05.119140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:05.126406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 242/493] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:05.126406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.140519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.226262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769172f1-8502-481b-8991-1d6cea763e7a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.241957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132acb7-73dc-413d-a93f-c7a3258c77b0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.243944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.265422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.266466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.266765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.267029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.276812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.277615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.277840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:05.277954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.278162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:05.286207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:05.287137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:05.287914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 252/494] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:05.288556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.292813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.294281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-86199c14-446a-4cb6-926d-8a0a692932ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:05.313083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:05.314216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:05.315073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:05.315073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:05.315073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:05.315073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:05.319618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 243/495] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:05.320872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80cfc4-1c6a-4a17-9b53-bfcf084bcf25 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.321125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80cfc4-1c6a-4a17-9b53-bfcf084bcf25 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.321345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80cfc4-1c6a-4a17-9b53-bfcf084bcf25 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.322787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 244/496] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:05.323410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80cfc4-1c6a-4a17-9b53-bfcf084bcf25 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.389162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.404571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7682f51-27fa-45f6-a8f2-61c4f4536b38 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.434862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80cfc4-1c6a-4a17-9b53-bfcf084bcf25 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.439656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.440330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.440610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.440884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.441726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:05.442038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.443504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.468410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.469075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.469353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.469674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.470417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:05.470723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.472104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d063a-bc40-409b-bb8c-da05d58cd2b6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.479297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 253/497] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:05.480082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 245/498] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:05.480298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.481145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c07d-2b4c-49bc-ba33-b15a3513a252 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.481428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c07d-2b4c-49bc-ba33-b15a3513a252 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.481703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c07d-2b4c-49bc-ba33-b15a3513a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.498089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2ca5c07d-2b4c-49bc-ba33-b15a3513a252 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:05.499295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2ca5c07d-2b4c-49bc-ba33-b15a3513a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:05.520459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.565588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 246/499] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:05.567128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63355bbf-4f92-4449-a403-8692c182cf29 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.567422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63355bbf-4f92-4449-a403-8692c182cf29 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.567690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63355bbf-4f92-4449-a403-8692c182cf29 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.569105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 247/500] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:05.569703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63355bbf-4f92-4449-a403-8692c182cf29 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.586729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb66c-4db5-45b4-8c48-788ec4585460 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.620148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.620898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.621187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.621466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.622196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:05.622541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.623943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.637349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 254/501] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:05.638161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.640163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.688956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63355bbf-4f92-4449-a403-8692c182cf29 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.722337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.723154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.723433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.729689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.734575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.736356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.737483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6a3f-aa38-436a-af79-0fcb79911284 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-bce228f5-7b2f-4549-ace5-bb669495d44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:05.754066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:05.755002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:05.755816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:05.755816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:05.755816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:05.755816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:05.755816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 248/502] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:05.756478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.756744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.757019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.773340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:05.774551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:05.774837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:05.775493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:05.775775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:05.776048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:05.839523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 249/503] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:05.840418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.843314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:05.845495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:05.890257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.939928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:05.952671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 255/504] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:05.952671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:05.965559 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f6ef-7c0e-4cd1-8edf-86d9c2fad510 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.004994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.005719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.005998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.006279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.017117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:06.024678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.025691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 250/505] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.026833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.062742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b098-f905-4c4b-bfc9-079e0dbaf665 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.095872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.096675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.096958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.097233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.104668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.106415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.121797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:06.122424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.123531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 256/506] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.125231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.133900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ed76-89ba-4f78-813a-d23e280527dd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.166581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.167237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.167512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.167786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.168533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:06.168835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.170181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.179889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.182241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 251/507] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:06.183266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.223363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96b96-768d-4ddc-92d7-da33540b2f55 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.255923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.256665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.256938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.257219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.267984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.269828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.281246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625f79c-717a-488e-b2b9-b875659e47e3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:06.287540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.288496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.289352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:06.289352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:06.289352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.289352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:06.289352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 257/508] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.289645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.318710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.319508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.319782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.320050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.330127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.331597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:06.349066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.350082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 252/509] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.355913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.359378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.390978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1d506-f99c-4e28-84a1-e1873edc9508 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.398645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.425134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.425800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.426081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.426357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.427130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:06.427431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.428820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.436606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 258/510] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:06.437366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.459419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b411-b4f4-4b58-9521-5dd1b8f5dd5f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.495856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.496621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.496978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.497261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.498119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:06.498476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.499920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.538048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 253/511] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:06.539000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.540270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b96782-850f-4da4-8bea-af3ac20d651d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.575291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.576096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.576382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.576655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.584874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.586338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:06.601028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.602049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:06.607533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:06.608198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.608281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 259/512] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.608281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.638767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.645146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555456a-d880-4a95-a5b0-7abc17c3e66e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.681748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.682412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.682716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.682993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.683786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:06.684092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.685528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.696954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 254/513] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:06.697744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.717249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474705fe-366e-4e16-8f77-c2097fd247f6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.750742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.751849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.752137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.752472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.753218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:06.753529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.755003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.767393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 260/514] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:06.768204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.797186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b861aa4-8ac7-430d-8131-7479a1bfbb3b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.820101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.831637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.832437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.832719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.832996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.843369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.844834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:06.856283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:06.866779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:06.867628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:06.868454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:06.868454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:06.868454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:06.868454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:06.868454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 255/515] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:06.868747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93b129-6950-4d86-b73f-dd4ada658f64 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.868814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:06.905976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:06.906639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:06.906941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:06.907249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:06.969813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423d0c3-5d72-42f8-80ba-f75b664e5b68 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:06.973705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:06.975184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.003305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.003958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.004238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.004509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.005349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:07.005652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.007099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.024587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 256/516] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:07.025426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.061455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.063326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.076665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 261/517] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:07.077467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.131285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376787b-98a7-4caf-ac5d-b69e2545ef80 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.166104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.166811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.167092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.167367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.168059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.169513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.176853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cfa26-2930-479b-8d6c-9b07539d1bb2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.189148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 257/518] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:07 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:07.189947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.214637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.215718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.216550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.217086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.225506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.227219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:07.243834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:07.244817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:07.245692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:07.250322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 262/519] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:07.251520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.290867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.306033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.306555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84168b-79fc-4993-afe3-715d09826b18 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.343347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.344738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.345209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.345541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.346273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:07.346556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.347993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.362122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 258/520] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:07.363180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.364376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf4609-6325-44ae-901d-f7dbc49e25b4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.399739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.400389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.400667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.400941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.473364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.473600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f5764-9dc1-445d-827b-57963a1b766c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.474868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.510785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.511577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.511882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.512152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.523256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.524704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.525254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:07.543982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:07.545040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:07.545892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:07.545892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:07.545892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:07.545892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:07.545892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 259/521] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:07.546442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.548703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.591195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 263/522] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:07.592210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.656830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb21b-4471-46e7-ba7a-d869ccc72b2a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.695681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.696371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.696643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.696920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.697721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:07.698032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.699499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.712563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec342ae-f530-4031-a357-7d31a9f81faf None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.724222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:07.724222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.756584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.757684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.757997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.758360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.767278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.768814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:07.783835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:07.784944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:07.786160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:07.789754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 264/524] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:07.790586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.793615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.801625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:07.840712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccd08-d3f7-47aa-933b-6c41d52f2b23 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.878906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.879582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.879866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.880139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.881296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:07.881804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.883149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.894321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd8bc-3ae9-4f57-8d06-479b71849dfb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:07.899310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 261/525] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:07.899310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:07.935627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:07.936429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:07.936710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:07.936982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:07.949115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:07.950645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:07.969584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:07.970589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:07.971433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:07.971433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:07.971433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:07.971433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:07.971433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 265/526] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:07.972244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.008253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63f6a-563a-4c3f-bec1-5725a7cf14cc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.035337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.039517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.049839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.050536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.050816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.051109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.051936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:08.052246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.053725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.059778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 262/527] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:08.060542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.092796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0944b6-0b49-4ffc-9794-57611b1bbc65 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.132090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.133281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.133562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.133838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.135073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:08.135384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.136874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.157668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 266/528] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Jul 4 04:02:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:08.162411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.206331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c546d56-9905-47c9-9573-e1bb0567d2b4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.254471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.255390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.255798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.256147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.263770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.265309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.280467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c48f2-6e50-4607-8aa3-f7f87c02b9fb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:08.283090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:08.284099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:08.285155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:08.287193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 263/529] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:08.288183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.316404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.341555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.342301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.342805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.343185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.344017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:08.344625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.346676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.350498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.364704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 267/530] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:08.365968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.419874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa51a9-9c92-410d-88f8-9b3cb36fd0ac None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.472023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.473071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.473503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.473942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.481965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d7a81-9000-45b7-a997-2d27a1d72eaa None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.520433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.521180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.521568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.522000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.522983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:08.523417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.524944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.550405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 268/531] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:08.550405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.554948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.556688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.587771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.609437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.659264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c867469-4c45-40fa-b66a-8facb5cddf75 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.670723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 264/532] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:08.671733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.694775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.695579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.695872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.696363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.703119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.704744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:08.718812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:08.720059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:08.720920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 269/533] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:08.721380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.785537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10054752-f161-4643-8281-61cb9a9acc16 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.823792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.824700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.825119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.825579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.826769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.827759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=dcab788fe9b0499c99336de1c06c885d) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:08.828244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.828620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08120-4368-4d06-9a9b-8a87252dd57e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:08.830700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.842768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 265/534] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/role_assignments?user.id=dcab788fe9b0499c99336de1c06c885d&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:08.843957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.866728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:08.870742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:08.871768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:08.871927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:08.872145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:08.899068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:08.900618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:08.936602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:08.937679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:08.939100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:08.939100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:08.939100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:08.939100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:08.939100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:08.947463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:08.971140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955988b-beaa-48a9-b89e-31685dae6c7a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.008808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.010603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.011070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.011934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.012960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:09.013454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.015114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.030429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:09.037797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.122282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:09.136010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa83cf-4096-41fd-8956-22f916445526 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.182007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.183951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.184242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.184525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.185271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:09.185584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.187055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.192727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a138a5-8755-4f39-9bda-ed8c9d216bb8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.218325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:09.220875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.239884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:09.257746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.260306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.263863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.264397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.266628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=0f9c5095b2ac44c1833141d0949c6ef6) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:09.267334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.270314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238432b-7f08-4eb9-b4c4-c01085393440 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.281468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 267/538] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/role_assignments?user.id=0f9c5095b2ac44c1833141d0949c6ef6&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:09.300819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd092d2-9563-41f4-b6e3-d02fff88b93c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.301129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd092d2-9563-41f4-b6e3-d02fff88b93c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.301374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd092d2-9563-41f4-b6e3-d02fff88b93c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.303019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 268/539] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:09.309304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.309599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.309843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.327507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:09.328813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:09.407148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 269/540] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:09.424569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca663e-e0f2-4052-8b21-ecd916a7063e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.438916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.489044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.490321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.490898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.491186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.501911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.503447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:09.522852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:09.523729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:09.524538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 272/541] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:09.529917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.559388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:09.618806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:09.625668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3de8-90ce-4772-90b9-1daecdfdadb3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.641668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144dd44-95a6-4652-81f3-9a19100163e8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.661352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.662161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.662479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.662769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.673419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:09.681306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:09.682218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:09.683208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 270/542] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:09.685499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.686183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.686525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.686810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.687706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:09.688027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.688757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.690336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.728463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 273/543] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:09.734309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.793923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5b2a-9f0c-467f-9de2-5624b98ce424 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.824008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:09.837281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:09.837281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:09.837281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:09.837281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:09.837732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:09.837732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:09.839160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:09.850277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 271/544] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:09.856834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:09.927875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14219e3a-34ad-4994-bf26-1fcd901974c6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:09.994747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:10.001620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.002722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.005415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.005844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.006948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=7e468089828e4180b8636e167d16cf8b) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:10.010120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:10.011773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:10.022926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 274/545] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/role_assignments?user.id=7e468089828e4180b8636e167d16cf8b&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:10.053529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6e60-070a-4add-9567-3510276ebe35 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:10.107067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.115669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.116185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.116659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.131369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:10.133169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0832067e-b498-4724-a86f-4e30a5f00f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:10.160871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:10.161791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:10.162730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:10.162730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:10.162730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:10.162730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:10.165008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 272/546] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:10.174471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:10.330793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37669fa-d3b4-4735-bac3-26f6b2d54e66 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:10.367343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661312a4-23a3-48c2-91a9-0cda4a137914 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.367995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661312a4-23a3-48c2-91a9-0cda4a137914 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.368407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661312a4-23a3-48c2-91a9-0cda4a137914 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.370244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 273/547] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:10.375677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.376385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.376981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.383858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.385897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.386407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.386869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.387781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:10.388266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:10.389822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:10.398523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:10.399444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:10.406130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 275/548] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:10.411765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:10.461974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:10.499915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:10.538485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 274/549] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:10.547941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7533f7-4cb8-42ba-8a0f-9b48af139e67 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:10.573375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:10.597502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.598455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.598671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.598948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.614340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:10.616015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:10.659039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:10.660556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:10.661619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:10.666841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 276/550] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:10.672205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:10.700781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead7a6-bfaf-4632-a4f5-b3c04afeb1ba None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:10.747191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.748011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.748293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.748566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.767905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:10.785596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:10.786599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:10.787398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 275/551] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:10.793034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:10.835613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:10.885361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635f954-cd83-49da-9f20-60f525678fa5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:10.914776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:10.955854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:10.957283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:10.958796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:10.959225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:10.960221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:10.962633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:10.964231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:10.978430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629218-6496-47c2-8882-85a8759a6abf None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.000263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 277/552] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:11.005841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.047262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.048240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.048760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.050170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.051253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:11.052848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.054507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.073222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 276/553] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:11.074703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.172302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.184321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab62da-3a47-4996-9bd7-1cbca0e230e5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.193271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06423-8c9d-4b7b-a9e1-1f2d7ef5d824 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.225893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.226733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.227014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.227287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.247552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.249012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.251197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.252045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.252331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.252608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-37d3c8e9-dfb0-451a-85c6-1055cd145137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:11.275264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:11.276173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:11.277463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:11.277463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:11.277463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:11.277463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:11.277463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 277/554] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:11.278879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974d9d-50f6-44fb-bddd-005fb20a8104 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.278879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974d9d-50f6-44fb-bddd-005fb20a8104 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.278879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974d9d-50f6-44fb-bddd-005fb20a8104 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.284497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 278/555] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:11.285401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974d9d-50f6-44fb-bddd-005fb20a8104 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.296688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.324063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.325562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030ab93-73c6-4b03-a525-4b7519640f7b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.397670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974d9d-50f6-44fb-bddd-005fb20a8104 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.432374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 278/556] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:11.434188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.434639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.435076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.441984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.442607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.442938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.443216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.443953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:11.444261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.445636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.454267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:11.456252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:11.459075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 279/557] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:11.460361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.462972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.546392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 279/558] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:11.546392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.547203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.608288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934405b1-f608-4f23-8cff-b0a558a76745 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.657454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.658479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.658863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.659235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.672148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.673799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.674126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930981f-f366-4696-8f33-5b660cd61917 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:11.699447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:11.700609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:11.701708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:11.701708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:11.701708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:11.701708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:11.704358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 280/559] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:11.705276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.715836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.716772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.717156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.717527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.726205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.727974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:11.742264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:11.743278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:11.744341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:11.746596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 280/560] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:11.747303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.771678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.816373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:11.821938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c526b-9191-4d7a-bd5b-5459bd075fff None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.857424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.858332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.858715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.859201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.862652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a49f1-f46c-4e06-869f-27dd8c56459c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:11.871037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4061dea8-363b-4b31-8bdd-e2561048cc96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:11.880004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:11.881133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 281/561] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:11.887376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff594-0b95-492a-b6e2-4db5f654aa53 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.888198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff594-0b95-492a-b6e2-4db5f654aa53 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.888198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff594-0b95-492a-b6e2-4db5f654aa53 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.888198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 282/562] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:11.888198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff594-0b95-492a-b6e2-4db5f654aa53 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.909704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:11.910515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:11.910867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:11.911224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:11.912035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:11.912419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:11.913871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:11.927648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 281/563] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:11.928654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:11.995220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff594-0b95-492a-b6e2-4db5f654aa53 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.010211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.032640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.033502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.033789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.034069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.034266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8084f-e1e3-4b2c-907d-ad1dd56a1e95 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.034949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.035268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.036710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb54151-ed6e-4ddb-b4e5-d2a806984a8d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.055362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 283/564] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.056712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.057000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.057289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.063387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.074427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:12.075616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:12.076606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.077289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.077749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.078029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.080313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.080658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.082721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.092206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 282/565] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.093021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.159480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 284/566] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:12.160457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.213983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332360c-1489-4bb6-8578-b0ec845354bc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.249891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.250832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.251068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.251492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.258046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.259807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.267437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06923ba-c083-470d-b576-78cb383ddbe5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.269144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:12.276107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:12.276988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 283/567] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:12.278469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.310054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.310919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.311193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.311473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.324183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.325546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.326231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:12.344746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:12.345843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:12.346869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:12.346869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:12.346869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:12.346869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:12.346869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 285/568] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:12.347633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.389032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6f4d-8069-4089-8ef8-179119a2bf34 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.427832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.428637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.428921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.429197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.440385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.442048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.457369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b612e-aefb-4b94-9a2e-50aea3006014 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:12.459901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:12.460937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:12.461843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:12.461843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:12.461843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:12.461843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:12.461843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 284/569] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:12.462162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.492203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.492996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.493278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.493553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.503956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:12.511976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:12.513058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 286/570] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:12.519789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.534473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.555307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.561652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938b4ad-2e61-4b12-a232-17851bc2d032 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.595825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.596486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.596760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.597029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.597819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.598123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.599598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869412aa-2d6f-427c-a90b-91367639e023 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.610914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 285/571] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.612105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f3b0-38d8-4dee-97a8-db3f121fbf02 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.612385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f3b0-38d8-4dee-97a8-db3f121fbf02 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.612655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f3b0-38d8-4dee-97a8-db3f121fbf02 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.614078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 286/572] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:12.614816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167f3b0-38d8-4dee-97a8-db3f121fbf02 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.623936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ea1a-ddac-4df4-9cc8-152f320598b9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.660778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.661455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.661742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.662018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.662778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.663088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.664478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.676976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 287/573] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.677759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.724809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167f3b0-38d8-4dee-97a8-db3f121fbf02 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.752361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.757507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.758119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.758430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.758718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.760031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.760141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.761439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.773762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 287/574] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.775038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.777141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0753a-dd77-4f73-9e39-45f19c59f2a5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.792760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:12.810398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.811077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.811358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.811633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.812392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.812701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.814254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd679-89b2-4119-b8d5-27ba59352b85 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.822217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 288/575] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.823422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.823710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.823988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.840122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:12.841322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:12.872848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b34e31-0e6f-41a4-a0d6-1ed14554f4f9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:12.906112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 289/576] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:12.907145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:12.920173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:12.921008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:12.921655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:12.921944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:12.923195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01612217d9fb486daccdc549d9e0c6fb) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:12.923519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:12.925426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:12.932102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 288/577] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/role_assignments?user.id=01612217d9fb486daccdc549d9e0c6fb&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:12.932753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.000187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.036805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.041735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0730b7-a3d9-4481-b2fc-5f495866c7d3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.059299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa3731-fbaf-4aa8-ab71-31f280dbeee3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.080546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.081362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.081649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.081930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.089930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.091523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.096846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.097654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.097942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.098216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.105983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.107475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:13.107718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:13.108757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:13.109820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:13.116249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 290/578] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:13.116249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:13.122111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:13.123079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:13.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:13.127984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 289/579] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:13.128825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.217799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5079002-d57d-4b89-92b8-a448e935ed32 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.236717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4bc77-cc5c-4727-b620-a188e48b8b7f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.243454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.252605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.253448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.253734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.254012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.266059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.267601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.276410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.277449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.277892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.278346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:13.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:13.286080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:13.287237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:13.287237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:13.287237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:13.287237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:13.289089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.291188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 291/580] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:13.292206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.293050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:13.301487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:13.302583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:13.303399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:13.307625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 290/581] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:13.308491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.402111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b89f-fa59-4248-84b6-0afd1397636e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.413867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905967b-bbf6-40b7-8917-4c15f661a773 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.437865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.439227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.439513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.439792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.440592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:13.440909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.442337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.458158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 292/582] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:13.458158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.467910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.468715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.469007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.469349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.470808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:13.471131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.472586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.492601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 291/583] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:13.493245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.504683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.560941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.568670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9118-849f-44ee-abd6-1238592dd223 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.604172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.604380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868167e-c638-4982-b64b-2401b3de7c87 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.604839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.605120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.605393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.606125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:13.606457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.607826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.624473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 293/584] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:13.624473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.649766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.650483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.650911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.651320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.652056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:13.652522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.654491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.664713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 292/585] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:13.665635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.727377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.730218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a924-19cb-4201-99d3-a6532a9c970e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.767494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.768393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.768678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.768956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.773254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22b090-c2e7-4c92-a73f-1ea51f57e869 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.783687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=789dd1683eac49d9b26609eb2903475e, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.785159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.795327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:13.802892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 294/586] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:13 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:13.810886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.811863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.812145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.812483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:13.815356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.827250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:13.829162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:13.849069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:13.850206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:13.851329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:13.851329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:13.851329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:13.851329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:13.852983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 293/587] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:13.853966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:13.957082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be0db8-6278-4c08-9139-7f4b9fd9a093 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.977474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde674a1-b2e9-4996-b708-719909f54274 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:13.994352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:13.995376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:13.995861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:13.996232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.004935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.006664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.015198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.016304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.016743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.017160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:14.023916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:14.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:14.025856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 295/588] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:14.027129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.033271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.035189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.050740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:14.054158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:14.055064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:14.055934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:14.055934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:14.055934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:14.055934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:14.058131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 294/589] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:14.059210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.086742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.153468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828199-70b4-4c96-a63a-43604460b062 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.166283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c888-45fd-493d-bdf8-1de30ed8bbaa None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.190230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.191035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.191320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.191597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.192483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.192784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.194255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.202814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.203865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.204350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.204822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.205859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.206398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.206810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 296/590] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.207667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.208485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.226554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 295/591] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.227720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.294794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.310302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.319117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073855d-5600-4c06-8be0-287935a51802 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.342044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad871f-877d-4a69-902d-86434a4a4c47 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.355402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.356383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.356830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.357245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.358111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.358693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.360483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.378256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 297/592] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.379517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.392065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.392975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.393263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.393539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.403185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.405153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:14.424074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:14.424980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:14.425899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:14.429959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 296/593] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:14.431361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.496330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0d69-bc5c-47ac-ae7d-b99f841e90d9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.526815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.537266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.538308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.538796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.539295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.550730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.552503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.565354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5e1d3-4956-490a-be2f-35f66586f9c0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:14.572071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:14.573131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:14.574152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:14.574152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:14.574152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:14.574152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:14.580845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 298/594] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:14.581915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.600729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.601514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.601739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.602014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.609831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.611448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:14.626988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:14.628133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:14.629146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:14.635004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 297/595] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:14.635388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.638191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.704268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da8759-d75c-40ec-a871-6be1ee22c5d8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.736187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf26d4-471a-40ec-96ae-736b6761e634 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.742352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.743224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.743673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.744193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.745114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.745559 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.747336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.761048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 299/596] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.762201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.776324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.777068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.777586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.777748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.778569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.778945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.780405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.792551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 298/597] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.797707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.812224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.858492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:14.879522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08592a-0d53-4096-9020-70786c115465 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.900982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53fdb-2d8c-4522-a971-374a727e6919 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:14.918274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.919304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.919850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.920258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.921233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:14.921679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.923583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.936814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:14.937798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:14.938206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:14.938565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/dcab788fe9b0499c99336de1c06c885d/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:14.945318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 300/598] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:14.946536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:14.959842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=dcab788fe9b0499c99336de1c06c885d, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:14.961406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:14.982462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 299/599] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:14 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/dcab788fe9b0499c99336de1c06c885d/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:14.984027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.084477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.091204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac4ef-bd6d-4d53-a3cf-cb01186df375 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.110362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09c8-4c4d-405f-bb34-a30c3a56559c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.126401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.143017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.144366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.145199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.145752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.155082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.157297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.158593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.159438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.159805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.160088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.173327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c4461a0e-d4e2-4f44-aac7-a76ebcc5541c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:15.174910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:15.175971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:15.177060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:15.177527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.179573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 301/600] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:15.181401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae96be-72ba-4c24-99b5-78e4d21a1069 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.181859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae96be-72ba-4c24-99b5-78e4d21a1069 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.182325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae96be-72ba-4c24-99b5-78e4d21a1069 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.184148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 302/601] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:15.190383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.190383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.190592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:15.203396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:15.204355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:15.205213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:15.205213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:15.205213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:15.205213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:15.205213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 300/602] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:15.206125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.213411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:15.214716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:15.310445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 303/603] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:15.314266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.340431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda11b-f750-4c9c-82b2-d7be119da054 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.388949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.389810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.390101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.390403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.391506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:15.391826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.394166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.410033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 301/604] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:15.410923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.416292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.435271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b9b1a-001a-4328-8a88-b7392f90cf92 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.451077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.472293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.473126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.473498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.474042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.474723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:15.475155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.476603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.489380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 304/605] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:15.490439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.542904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c11e0a-3731-4576-8b56-2b74990b4b28 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.579231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.580058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.580341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.580627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.591877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:15.596197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04f46-a451-4333-a487-ed2ac9b8841e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:15.599734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:15.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 302/606] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:15.602213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.638414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.639790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.640114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.640826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/0f9c5095b2ac44c1833141d0949c6ef6/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.660804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=0f9c5095b2ac44c1833141d0949c6ef6, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.662610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.673843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.686465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 305/607] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:15 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/0f9c5095b2ac44c1833141d0949c6ef6/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:15.687748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.734055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b3e13-48fd-4691-88e1-089d1f593437 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.745016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:15.823290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.824133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.824335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c646-ac77-48ae-a3e2-af5775662ad9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:15.824537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.824822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.839746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.841492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.865092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:15.865785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:15.866071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:15.866348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:15.866465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:15.867265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:15.868112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:15.868586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:15.868586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:15.870210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:15.870545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 303/608] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:15.877150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.880925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 306/609] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:15.888443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:15.962919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.005926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4775-14e8-41a7-bd5b-0944e1e3a35d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.010836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a22a3-40cf-42f8-81d2-f1aaa0468bf1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.047386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.048037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.048419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.048748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.049206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.049667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:16.049985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.050487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.050919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.051328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.051471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593136f8-376c-4451-84d2-714c7909a7fa None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.062556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.064197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.069601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 304/610] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:16.071336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b95-9aeb-4039-96b8-f159ccbb32fb None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.071458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b95-9aeb-4039-96b8-f159ccbb32fb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.071683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0b95-9aeb-4039-96b8-f159ccbb32fb None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.073316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 305/611] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:16.073991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0b95-9aeb-4039-96b8-f159ccbb32fb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.082132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4b2dbfeb-7e49-45a4-88df-e199c72ac61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:16.083935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:16.084925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:16.085865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:16.085865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:16.085865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:16.085865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:16.087792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 307/612] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:16.089231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.089689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.090130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.107033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:16.108487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:16.190468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 308/613] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:16.192052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.205146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0b95-9aeb-4039-96b8-f159ccbb32fb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.253129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.254022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.255560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.255838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/7e468089828e4180b8636e167d16cf8b/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.256196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.273892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=7e468089828e4180b8636e167d16cf8b, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.275425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.298406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 306/614] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:16 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/7e468089828e4180b8636e167d16cf8b/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:16.298406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.366350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.412511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68ccf-72e3-4a13-8c8f-eae40b8ba691 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.412716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694947cb-99c1-412f-8e5d-d169c6663819 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.453257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.454168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.454484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.454752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.467688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:16.476355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:16.477276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:16.478129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 307/615] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:16.488647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.503499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.504414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.504836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.505263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.506128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:16.506601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.508128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.524985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 309/616] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:16.530216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.667870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e05764-59e5-4bed-a23d-011ae39ffa1b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.682537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.704473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.705399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.705822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.706238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.707160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:16.707632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.709159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.718226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 308/617] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:16.720626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a405-e0ff-4b74-8fef-060114a64a89 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.723834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.726675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:16.799753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.800491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.800780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.801064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.823581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.825051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:16.851553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:16.852748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:16.854069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 310/618] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:16.861649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:16.868593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549613-39d5-43fa-931b-bba9137cb06c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:16.947598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:16.948580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:16.948975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:16.949359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:16.983819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:16.986116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:17.027505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:17.028700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:17.029921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:17.029921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:17.029921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:17.029921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:17.032956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 309/619] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:17.038168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.077626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583cf08-61f9-4ced-98b9-70f4f964a7f0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.101972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.150743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.152470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.152756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.153036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.153907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:17.154217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.159110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.200896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 311/620] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:17.208280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.253132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.279728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e746ce-9e8a-4f97-9b49-d965c5954e70 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.352539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.353592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.354093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.356514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.357492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:17.358031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.359650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f315223-bd55-41f9-b1a4-45e08e28d638 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.380759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 310/621] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:17.383027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272e23-ea77-4b35-afeb-cec9ae1b17a7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.383708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272e23-ea77-4b35-afeb-cec9ae1b17a7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.384718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272e23-ea77-4b35-afeb-cec9ae1b17a7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.386830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 311/622] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:17.387882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272e23-ea77-4b35-afeb-cec9ae1b17a7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.401777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7373e-403f-4359-a729-37997bd3a093 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.437181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.438416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.438980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.439467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.450288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.452040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9a69fd1f-3e75-4428-b6ee-7bd46aebd593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:17.471782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:17.472737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:17.473649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:17.473649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:17.473649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:17.473649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:17.481576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 312/623] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:17.483122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74128627-0957-4879-ac83-561c950507b7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.483765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74128627-0957-4879-ac83-561c950507b7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.484256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74128627-0957-4879-ac83-561c950507b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.503550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-74128627-0957-4879-ac83-561c950507b7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:17.505085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-74128627-0957-4879-ac83-561c950507b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:17.595247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.612502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 313/624] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:17.613843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74128627-0957-4879-ac83-561c950507b7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.616417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.649290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272e23-ea77-4b35-afeb-cec9ae1b17a7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.688945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.690001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.690294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.690606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.700875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.702704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:17.723707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:17.724709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:17.725654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:17.730086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 312/625] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:17.731188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.745044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74128627-0957-4879-ac83-561c950507b7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.790601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.791869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.792340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.793042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.794469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:17.795265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.797125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:17.814717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 314/626] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:17.818446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.866570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46b5ee-f0f5-4905-b61b-80fd6542a2a8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.895402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.911833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.912638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.912916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.913190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.926854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:17.937446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:17.938519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:17.939385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:17.942513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 313/627] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:17.943878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:17.949461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b4f1e-1ada-4176-bb00-4317d6f76581 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:17.959389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:17.993702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:17.994754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:17.995219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:17.995674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:17.996722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:17.997597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:17.999609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.022869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 315/628] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:18.024263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.078296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22e17c-35a8-4932-bd68-549b0e019a6f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.122839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.123681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.124030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.124404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.133127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.134937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.147926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62a247-e1db-4815-a9ac-22b3c51ccb51 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:18.149790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:18.150795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:18.151710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 314/629] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:18.152226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.174943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.191770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.192445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.192725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.193044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.194216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:18.194555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.196355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930658-fb89-422c-b00b-671a8750a667 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.207864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 316/630] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:18.209177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c690b99-7bf4-407b-921c-30c4473e9cd9 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.209512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c690b99-7bf4-407b-921c-30c4473e9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.209792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c690b99-7bf4-407b-921c-30c4473e9cd9 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.211755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 317/631] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:18.212685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c690b99-7bf4-407b-921c-30c4473e9cd9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.223445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.260086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084c804-89ec-409b-a94d-7535b5935ff7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.297865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.299344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.299838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.300311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.326276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c690b99-7bf4-407b-921c-30c4473e9cd9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.361368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.362041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.362351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.362667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.363464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:18.363773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.365207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3017ba-e2d1-4bf3-bce1-f3fa5468173d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.377970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.378243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 318/632] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:18.379517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.379801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.380074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.380571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.399341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:18.400758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:18.438321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.451250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.478406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 319/633] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:18.478406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.500098 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 315/634] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:18.501508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.584240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa1259-3926-4b61-b9cc-1f3f1245935c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.599474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bf492-6c53-4007-9c04-4798878afb91 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.619056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.619869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.620151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.620413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.633617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.635446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.637603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.638537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.638816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.639090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01612217d9fb486daccdc549d9e0c6fb/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.653397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=01612217d9fb486daccdc549d9e0c6fb, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:18.653540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:18.654617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:18.655530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:18.655530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:18.655530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:18.655530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:18.655530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 320/635] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:18.655834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.655915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.671390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 316/636] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:18 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01612217d9fb486daccdc549d9e0c6fb/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:18.672468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.694681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.698080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.785336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e7ab0-8785-4729-bbf5-3075927325fa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.793810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cabb74-8032-45d0-ae57-c850dae8da95 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.833853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.834703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.834993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.835270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.835976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:18.836770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.837063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.837339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.847260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:18.848579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:18.850147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-84ed9872-14f2-420f-a77d-64673b78ce3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:18.855194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:18.856173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:18.857016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 321/637] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:18.858240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6193e-b5f4-4246-bbaf-33d530629d52 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:18.858337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6193e-b5f4-4246-bbaf-33d530629d52 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:18.858621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6193e-b5f4-4246-bbaf-33d530629d52 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:18.860017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 322/638] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:18.860767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6193e-b5f4-4246-bbaf-33d530629d52 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:18.873718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:18.874695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:18.875765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:18.875765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:18.875765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:18.875765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:18.875765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 317/639] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:18.876043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:18.944916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.959540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:18.978424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6193e-b5f4-4246-bbaf-33d530629d52 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:18.978740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b6ceb-3ee4-4d9a-9a86-33bbd3238ca3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.015209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.015884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.016215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.016490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.017243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.017549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.018944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aac3-29fb-4dec-b798-678a1f3c4cec None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.020825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.021491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.021767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.022038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.022785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.023089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.024556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.034864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 318/640] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.034864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.034864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.034864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.039976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 323/641] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.044404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.050130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:19.051513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:19.120629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 319/642] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:19.121572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.155280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e878d-88a2-40d2-b87f-c4e0bbd37f28 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.191342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.192008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.192288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.192558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.193198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.194088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.194388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.195851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.213604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 324/643] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.214219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.218722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.236683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c31d4-fbc9-421b-a0d0-8229c99f3c66 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.275828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.276641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.276920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.277321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.289690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.291173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:19.308900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:19.309940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:19.310898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:19.310898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:19.310898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:19.310898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:19.310898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 320/644] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:19.311200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.326283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe531c2-03f7-4797-ba41-7ad3175c97fa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.366441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.367367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.367652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.367924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.377386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.378852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:19.394467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:19.395479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 325/645] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:19.397078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.420391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c941-cfe8-4018-9b6e-b101f41d98ef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.435361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.458113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.458853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.459142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.459422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.467748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.471203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:19.478910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:19.479818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:19.480635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 321/646] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:19.481346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.511602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d750c4-9580-46ab-a9e4-2cc9631043a2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.547598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.548534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.548804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.549095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.556615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.558192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:19.573402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:19.574292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:19.575224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 326/647] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:19.575704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.585027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398dfb2-e3bd-422a-b5db-98fcf822ec79 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.620339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.620997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.621277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.621549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.622305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.622648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.624010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.641554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 322/648] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.642300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.679233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.689723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb7e3f-df24-493b-8837-19cef5cda997 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.715358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.727150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.728264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.728548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.729113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.730241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.730588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.732488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.750461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb3880-ee8f-4809-b321-a8ad8a615453 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.752177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 327/649] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.752849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.793365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.794173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.794468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.794739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.795485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.795895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.797326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.810456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 323/650] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.810456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.860288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d86679-7074-4344-9571-c1367dca0146 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.896251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.896916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.897197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.897471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.898267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:19.898605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.900032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.904855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.912142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e27b06-61bd-4ccb-9855-b50c74ccdec7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:19.917541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 328/651] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:19.917541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:19.949576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:19.950567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:19.950824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:19.951114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:19.959127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:19.960576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:19.962799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:19.977823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:19.978793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:19.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 324/652] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:19.980980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.026288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd438c-da6f-4adc-8ce4-b4edb1ceee05 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.059558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.060212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.060493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.060748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.082635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdffc23-9936-4dee-af57-60e366104b24 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.117342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.118251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.118569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.118974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.128274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.129729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.129835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.131293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:20.150001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:20.151055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 325/653] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:20.157028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.167028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.184919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.253608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 329/654] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:20.253608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.267674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48c56-9173-4d5b-afc3-e62db6c12c2e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.308709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.309420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.309699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.310211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.310847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:20.311164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.312673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.330418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 326/655] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:20.330418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.374345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e0657-0f24-4206-a9dc-142bc5fdca7b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.416589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.417368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.417718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.418062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.419043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:20.419441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.420980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.423451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.439419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.448609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 330/656] 10.222.0.16 () {56 vars in 1137 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:20.449653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.464330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae1706-8203-4ea1-bbab-0a39e552da11 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.502072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.502890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.503268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.503671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.504426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:20.505222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.507860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.526476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 327/657] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:20.527423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.586292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0a2c-c4f6-45c6-bbcc-a6df9d6756fe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.631801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.632762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.633135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.633751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.647655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.649565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.649950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c235a0c-79ce-498c-a613-174bd8f9baab None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:20.675177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:20.676331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:20.677427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:20.677427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:20.677427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:20.677427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:20.680223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 331/658] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:20.681459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.683539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.690680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.691593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.692041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.692470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.743289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.764039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.765413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013fdf8-e261-4da2-ba67-3cfcbf4c6219 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.795964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aedf3-e20c-4276-8d0c-91800f3926e1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.831668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:20.832474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.832757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.833035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.841451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:20.843359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:20.858906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:20.859914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:20.860848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 332/659] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:20.861465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.873330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 328/660] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:20.874505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103311-0031-4bf8-b602-396e72bc80bf None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:20.874796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103311-0031-4bf8-b602-396e72bc80bf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:20.875073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103311-0031-4bf8-b602-396e72bc80bf None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:20.880957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 329/661] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:20.880957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103311-0031-4bf8-b602-396e72bc80bf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:20.915959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.974576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df240e-7c7a-4fcf-bf58-96084a78c36d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:20.989975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:20.991867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103311-0031-4bf8-b602-396e72bc80bf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.013015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.013689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.014485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.014732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.015450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:21.015764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.017198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.029184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 333/662] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:21.030437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.035060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.035960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.036306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.037459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.038329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:21.038694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.040136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511b8b1-6fef-4122-a93b-3c82f47eac08 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.059451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 330/663] 10.222.0.16 () {56 vars in 1139 bytes} [Mon Jul 4 04:02:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:21.060571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.060863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.061143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.077809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:21.079086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:21.162425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 331/664] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:21.162425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.162761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d144d7a-0e85-4a17-b53d-c17679707d51 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.183633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.197267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.198147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.198519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.198860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.209829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.211313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:21.229174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:21.230102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:21.231518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:21.231518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:21.231518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:21.231518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:21.235171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 334/665] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:21.235856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.263452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.271855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7408a-9e9b-4144-ac11-2b6b1eb380af None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.307579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.308388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.308671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.308948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.315996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.317456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:21.331250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:21.332124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:21.333012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 332/666] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:21.334209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.345650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63e22-ca85-407f-8146-f7a4126595c0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.381140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.381787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.382066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.382569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.418344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.438442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857eaedd-e7ef-4e25-a7ab-05abced60c90 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.453619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.455205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.474945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.475753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.476038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.476318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.481108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.488100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:21.495810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:21.496725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:21.497962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 333/667] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:21.498896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.568490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 335/668] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:21.568490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.611094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7f7be-e9e3-4da5-81d6-d6c46ebdd539 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.647652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.648452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.648836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.649211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.650184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:21.650454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.651904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.662642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.667182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 334/669] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:21.668190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.682683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901ab9-6681-4206-9aac-3d15f6b568af None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.707288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.719460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.720135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.720423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.720794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.721509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:21.721823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.723346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.735610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 336/670] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:21.736401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.783073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5456a2-48ba-4e6f-8945-4909be197d54 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.825761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.826534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.826921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.827326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.829217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:21.829662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.831726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.840426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 335/671] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:21.840810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e6fbb-361c-405b-81d9-bf6b0ed9477c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.841394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.879741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.880794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.881078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.881351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.892775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.894203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:21.907725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:21.913210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:21.914094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 337/672] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:21.915745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:21.954151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3cfbf-1d56-40db-890a-6e750e6594a8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:21.955593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:21.987863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:21.988665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:21.988946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:21.989221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:21.996056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:21.997490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:22.012134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:22.013247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:22.014482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:22.018101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 336/673] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:22.018972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.029055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b59db-2dc8-4c40-9fb1-7d8a9afa8ad3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.066799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.067550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.067841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.068120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.068968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=dcab788fe9b0499c99336de1c06c885d) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.069283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.070788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.080950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 338/674] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:21 2022] GET /identity/v3/role_assignments?user.id=dcab788fe9b0499c99336de1c06c885d&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.082316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.156945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb46eb-cb85-4258-86bd-b985bdc54610 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.190055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.205327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.208933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ba5db-b714-453c-b674-3e9ab4bd71b9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.209866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.211142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.211586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.213130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.226042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.227817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:22.246922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:22.247896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:22.248785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:22.248785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:22.248785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:22.248785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:22.248785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 337/675] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:22.249357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.262132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.263312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.264191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.264191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.264809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.264882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.266954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.281842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 339/676] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.286400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.372130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef247-d8ce-482e-96b0-5fc75c287f8b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.392372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbce0d-3eef-4c3b-990d-8638d9c2c945 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.419553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.420469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.420764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.421047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.422290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.422656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.424635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.430087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.431027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.431481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.431924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.432844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.433331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.434884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.438666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 338/677] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.444104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.453822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 340/678] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.454950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.478347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.484671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.566301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b355-87f0-4e5f-8aa3-5ca7af9158ad None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.568761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6bf0-5c2b-4c9b-b31e-5f2651940573 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.606021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.606648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.606983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.607354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.607450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.607721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.607814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.607997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.608925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=0f9c5095b2ac44c1833141d0949c6ef6) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.609328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.611234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.614993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.616418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.622834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 341/679] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/role_assignments?user.id=0f9c5095b2ac44c1833141d0949c6ef6&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.624037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4bf99d7e-e466-44b6-abbd-a6435ad93efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:22.643007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:22.643815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 339/680] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661a81e-856b-49ff-b1b2-f51428de0aee None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.644685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661a81e-856b-49ff-b1b2-f51428de0aee None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.645275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661a81e-856b-49ff-b1b2-f51428de0aee None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.646329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 340/681] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:22.646528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661a81e-856b-49ff-b1b2-f51428de0aee None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.743947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.764364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404318e8-7878-46c2-8470-cce8566489ad None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.767907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:22.779737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661a81e-856b-49ff-b1b2-f51428de0aee None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:22.809233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.810136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.810542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.810964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.820462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.821987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.823386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:22.824282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.824644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.824992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.826349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:22.826757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:22.828248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a2e59e54-41b3-483a-9f07-bfa6562e33cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:22.839360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:22.840566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:22.841771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:22.842645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 342/682] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:22.843002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 341/683] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:22.843831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:22.844200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:22.844477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:22.848617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.866132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:22.867361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:22.964383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 343/684] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:22.967271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:22.983293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30e4e8-9f95-4aa8-ac02-13d48d833cc6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.021690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.022563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.022946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.023319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.024263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:23.024696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.026285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.045921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 342/685] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:23.046919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.062312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.077407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.103828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197eba7-0abf-428c-a0f8-a4fbea9a1077 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.142350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.143538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.144128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.144698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.146234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=7e468089828e4180b8636e167d16cf8b) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:23.146735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.148376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.162651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 344/686] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:22 2022] GET /identity/v3/role_assignments?user.id=7e468089828e4180b8636e167d16cf8b&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:23.163813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.176087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b943b2-60fd-457b-b1c0-53b5eccb80c2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.229809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.230588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.232034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.232343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.261474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:23.275976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:23.277011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:23.277852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:23.283915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 343/687] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:23.290626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.382780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedebaa-5014-4932-b659-de7041c3b5f3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.413465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.425004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.425670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.425954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.426233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.442732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.453767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbde5f3-8d67-4696-bd25-73bf11c86764 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.497003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.497821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.498112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.498420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.499189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:23.499663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.501790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.505409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.506965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.512487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 344/688] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:23.523639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.631947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e563bf-efc4-45d9-8157-b2dd97053557 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.676089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.676908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.677249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.677536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.686000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.695069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.696466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:23.726491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:23.727458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:23.728340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:23.728340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:23.728340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:23.728340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:23.738172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 345/689] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:23.739103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.799136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 345/690] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:23.804500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.868427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62bf8-ac9a-4883-8958-82d2430c9cb3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:23.878738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:23.927172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:23.927875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:23.928153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:23.928434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:23.929184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:23.929492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:23.930949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:23.960445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 346/691] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:23.966905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:23.984780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f4686-c5d4-4bd9-90e6-11366fa792ab None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.054571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.055298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.055647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.055930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.057094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.075763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.077213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:24.117758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:24.118733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:24.119700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:24.119700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:24.119700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:24.119700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:24.125417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 346/692] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:24.136108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.162273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db22a-3fc3-4559-93d7-d01c69461fae None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.227022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.228219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.228701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.229295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.245041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.246508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:24.271682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:24.272834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:24.273810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:24.278718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.282210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 347/693] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:24.285483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.294757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283964-a25e-4f54-b3e6-d8cafd708309 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.337411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.338000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.338276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.338582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.339674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:24.339986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.341593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.361060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 347/694] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:24.362293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.456172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2a58b-c656-49c6-9a78-a712878c6456 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.464103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.480348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3dc03-cb36-496b-a717-7f817861f26e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.493158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.493841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.494171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.494533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.495387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:24.495709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.497148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909f56e-290d-409e-9b1c-ac237889fcdc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.515345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 348/695] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:24.515615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.516473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.516694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d834-8800-41b9-9edf-f2a89901e52c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.516792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.516962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d834-8800-41b9-9edf-f2a89901e52c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.517060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.517243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d834-8800-41b9-9edf-f2a89901e52c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.518760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 349/696] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:24.521158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54d834-8800-41b9-9edf-f2a89901e52c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.529280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.530855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a854cca3-309e-4d1e-94b3-cf91f9752759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:24.549922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:24.550899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:24.551771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 348/697] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:24.552785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.553080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.553362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.574256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:24.575517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:24.580812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.659388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54d834-8800-41b9-9edf-f2a89901e52c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.674096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 349/698] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:24.674989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.706236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.706947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.707235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.707516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.758705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.779534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.781272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.799600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b10262-61bc-493c-a25d-3c8b4ed4da8f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:24.839697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:24.840366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:24.840654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:24.840934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:24.841735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:24.842055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:24.843575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:24.852462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.857061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 350/699] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:24.858890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.901710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 350/700] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:24.902770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:24.971310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:24.978036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b0e-9650-4143-84a8-24d24e6ccb89 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.011183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c9476-8164-4693-8777-dac7bc153a6b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.013540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.014347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.014660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.014940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.026016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.027623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:25.049261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:25.050245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:25.051825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:25.051825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:25.051825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:25.051825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:25.052053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.052053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.052214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 351/701] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:25.052281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.052281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.052409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.052478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01612217d9fb486daccdc549d9e0c6fb) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:25.052478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.053803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48552-d282-4c55-ab3e-824818de4450 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.066776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 351/702] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:24 2022] GET /identity/v3/role_assignments?user.id=01612217d9fb486daccdc549d9e0c6fb&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:25.070179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b431-faea-48c1-b019-1b47c6e9017e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.070179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b431-faea-48c1-b019-1b47c6e9017e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.070179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b431-faea-48c1-b019-1b47c6e9017e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.070179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 352/703] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:25.075038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b431-faea-48c1-b019-1b47c6e9017e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.117177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.176458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd0515-3f48-4fa8-9d99-2db423469a88 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.224393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b431-faea-48c1-b019-1b47c6e9017e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.225335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.227085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.229119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.229412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.244128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.246526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.263474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-440f4cd0-f591-480e-a534-228e9e5d2314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:25.275103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:25.276064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:25.276965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:25.276965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:25.276965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:25.276965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:25.276965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 352/704] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:25.278267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.278726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.279229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.281989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.283353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.283353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.283353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.284902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:25.284902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.285905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.300342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:25.300948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 353/705] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:25.302526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:25.302663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.395596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 353/706] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:25.396790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.447697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b843a12-2b90-427e-9c37-a7153e7d692a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.458332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.490071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.490775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.491060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.491337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.492068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:25.492376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.493745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.508831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 354/707] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:25.510086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.552886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6fd35-f5b6-417d-af15-3c7a3321b2a8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.569974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.594397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.594845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.606704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.607066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.608092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.609452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.630007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302373-c78c-4178-a81e-e32969f635da None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.649212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 354/708] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:02:25 2022] POST /identity/v3/users => generated 254 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:25.649212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.681006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.681909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.682194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.682503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.683393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.684908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-eb99fdd7-2af1-476f-92dd-7011405e22a9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:25.700628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:25.701700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:25.702644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:25.707137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 355/709] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:25.708727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5ad9-557d-4ce8-bd6e-93edb52372f4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.709015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5ad9-557d-4ce8-bd6e-93edb52372f4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.709299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5ad9-557d-4ce8-bd6e-93edb52372f4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.711097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 356/710] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:25.711896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc5ad9-557d-4ce8-bd6e-93edb52372f4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:25.789419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.806735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3b52-4b5b-486d-bfdb-29c9b0fd0865 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.855654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.856481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.856765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.857044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.863483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc5ad9-557d-4ce8-bd6e-93edb52372f4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:25.864499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.865954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.880762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-43c56da7-42ae-4dcd-a9da-82eefd2ee2ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:25.888885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:25.889896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:25.890846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:25.891961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 355/711] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:25.893804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.894270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.894785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.913985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:25.915143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:25.915365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:25.915849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:25.916327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:25.916661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:25.917483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:25.918008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:25.919669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:25.925484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 357/712] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:25.926778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.009781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 356/713] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:26.009781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.061923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bbce5-57c0-4102-b4a8-9842ba1b1d43 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.101437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.102286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.102824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.103207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.104262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:26.104678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.106237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.121186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:26.124269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 358/714] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:26.126190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.146640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa79a8-1f8c-4296-bf9a-8f6fef1fd14e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.164869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:26.185755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.186695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.187147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.188000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.188864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:26.189312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.190952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.197758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 357/715] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:26.198854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.241868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa309-047f-4e68-8c30-83ea2ae02305 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.281489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.282181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.282490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.282772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.283604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.285019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:26.298648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:26.299916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:26.300887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 359/716] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:26 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:26.309776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.317727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66db1-b916-4c06-9238-22cc1a10601e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.355159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.356116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.356538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.356983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.358063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:26.358549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.360366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.367918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 358/717] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:26 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:26.423742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:26.426816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1ea88-58de-4c48-8c52-40d7551ac9b7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.467274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.467981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.468269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.468549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.469335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] RBAC: Adding query filter params (name=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:26.469657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.471115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2d8f1-4d58-4f3c-97a1-20a2e5ebeb49 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.476755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 360/718] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:26.482547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:26.501485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:26.608779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3669a-fafe-4610-893f-6a8c955926b5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:26.651759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:26.652488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:26.652821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:26.653104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:26.653826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:26.654136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:26.655553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe8a6e-d92d-4859-8cda-98273d5c6ce0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:26.660857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 359/719] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:27.673784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210bc14-95c4-4774-905f-9497abf255f6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:27.674704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210bc14-95c4-4774-905f-9497abf255f6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:27.675138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210bc14-95c4-4774-905f-9497abf255f6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:27.676931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 361/720] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:27.682592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:27.683414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:27.683843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:27.706290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:27.707687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:27.808980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 360/721] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:27.852829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210bc14-95c4-4774-905f-9497abf255f6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:27.979728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210bc14-95c4-4774-905f-9497abf255f6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.024502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.025316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.025597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.025873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.045734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:28.053671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:28.054732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:28.055570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 362/722] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:28.062756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.064925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.228643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e726fa2-6b28-4c12-994e-b7bd7fd8eeee None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.276390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.277122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.277409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.277689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.278481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:28.278798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:28.280201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4a265-8829-4ced-9dc5-a5a298b2384e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:28.292215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 361/723] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:28.297688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.333228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.373347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc898e-bc8c-4f2f-963e-33740a4a8b50 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.373659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc898e-bc8c-4f2f-963e-33740a4a8b50 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.373922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc898e-bc8c-4f2f-963e-33740a4a8b50 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.375724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 362/724] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:28.382618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.382910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.383196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.410973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab363c72-6cea-40e5-a2bc-ea4f020dd9c4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.419543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:28.420753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:28.469950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.470948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.471234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.471512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.486486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.492589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 363/725] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 04:02:28.493180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:28.498401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:28.512873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:28.513765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:28.515332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:28.515332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:28.515332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:28.515332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:28.515332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 363/726] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:28.519750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.533025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.622792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.644155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e707de-1909-43fb-bc48-50baa99f5c81 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.680883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.682430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.682859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.683289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.695903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:28.706199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:28.707405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:28.708443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:28.711399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 364/727] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:28.717629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.784835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdce5a-28e8-4620-bbd8-e04baf085f27 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.841929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.843822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.844960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.846868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.848665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:28.851777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:28.853395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c387b-43b6-4f96-bd77-b169dd6d7472 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:28.870547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fed3f8-a71f-42db-b780-ad1ee03e25e9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:28.871442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 364/728] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:28.873008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84810605-b1c3-4d56-bb35-e396a7980c6c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.873611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84810605-b1c3-4d56-bb35-e396a7980c6c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.874043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84810605-b1c3-4d56-bb35-e396a7980c6c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.875993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 365/729] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:28.876857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84810605-b1c3-4d56-bb35-e396a7980c6c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:28.911541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:28.912220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.912500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.912769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.913525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:28.913825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:28.915228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e4a35-7e6f-4fea-b9d2-8aad2862d897 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:28.917551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.926534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 365/730] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:28.928803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15ca95-4567-48c9-8c53-ef8f15b685eb None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.929096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15ca95-4567-48c9-8c53-ef8f15b685eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.929374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15ca95-4567-48c9-8c53-ef8f15b685eb None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.934653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 366/731] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:28.934653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:28.934653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:28.934653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:28.945111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:28.949629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:28.950891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:28.992418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84810605-b1c3-4d56-bb35-e396a7980c6c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.025738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.026567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.026853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.027126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.038646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 367/732] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:29.038646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.039657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.041122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-95fbd2c5-4b72-4aec-950d-994bfb06c567 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:29.056005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:29.057173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:29.058391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:29.061754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 366/733] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:29.063345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.063732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.064011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.080526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:29.081693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:29.149991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 367/734] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:29.151546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.154622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da1b1-7927-408e-9dce-8a7e24215944 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.182346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.191827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.192633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.192913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.193185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.202712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.203971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.205579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:29.226418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:29.227389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:29.228290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:29.228290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:29.228290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:29.228290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:29.228290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 368/735] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:29.228617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.264718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f6356-aa91-41f3-a178-9e523afea9fb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.298603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.299325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.299615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.299924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.310787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:29.318445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:29.319417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:29.320349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 368/736] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:29.321037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.335236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4c944-acac-44fb-85a3-2ec1e00e9f96 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.372357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.373052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.373368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.373642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.374524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:29.374825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.376296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.394221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 369/737] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:29.395237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.433316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852a0f3-7291-43f6-8345-61d1a472219c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.436383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.438608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.468241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.468951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.469234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.469512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.470245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.471647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b71cf-b508-407e-a036-b8fbc9aab43e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.486948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 369/738] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:29 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:29.488212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9313221-bfa8-4575-9dd9-1e2bf244d86c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.488505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9313221-bfa8-4575-9dd9-1e2bf244d86c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.488795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9313221-bfa8-4575-9dd9-1e2bf244d86c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.490168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 370/739] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:29.490762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9313221-bfa8-4575-9dd9-1e2bf244d86c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.509128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b3f3-2750-4735-abbb-227734d71c21 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.557373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.557963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.558314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.558619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.559359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:29.559673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.561084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.574039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 370/740] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:29.579418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.618222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9313221-bfa8-4575-9dd9-1e2bf244d86c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.671441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.672158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.672508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.672787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.673552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:29.673947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.675368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d6d44-3dc0-4084-837f-3ffc56f41200 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.686251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 371/741] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:29.691432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.691719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.692274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.698777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.705798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40d549-8bc4-4797-8f80-0cc94143b20b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.711119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:29.712364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:29.733626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.742137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.742864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.743154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.743433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.785584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 372/742] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 04:02:29.787124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.822255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.823698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.907653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28141308-fc30-4f43-90ea-d1d443ae8fd9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:29.920744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:29.934414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 371/743] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 04:02:29.934414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.949937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:29.950903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:29.951185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:29.951589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:29.959655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:29.961099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:29.977036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:29.977990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:29.979009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 373/744] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:29.979586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:29.998067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.043814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586151e-5a50-43bb-88fc-53db048c53ef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.079667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.080488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.080773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.081029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.082985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a676321-9c05-430a-a39a-fff2cbcc1054 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.091555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.093112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:30.111913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:30.113041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 372/745] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:30.113929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.121627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.122300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.122612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.122888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.123610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.125082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.155268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 374/746] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:29 2022] POST /identity/v3/services => generated 240 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:30.155268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.183363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.251437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabbf8e-2212-4936-9140-e991fb59e53f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.262726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.303741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88343f60-23e0-420a-8b20-2ffc72f6550d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.310289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.311136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.311469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.311745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.327274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.329411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:30.350255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:30.351241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 373/747] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:30.352512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.359180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.359180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.359458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.360338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.361210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:30.361528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.364212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.387344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 375/748] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:30.388161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.475960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.484495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ec878-952f-4816-aed4-581391d0d3b6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.520105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3b5a-c9f3-40b4-a79d-e11e129579fa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.521240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.522024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.522395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.522816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.524121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:30.524121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.525460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.537874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 374/749] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:30.543327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.556810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.570072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.571218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.571647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.572467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.573610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:30.574297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.576513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.592755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 376/750] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:30.594826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.672088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38bb03-c7c5-4e31-a59e-17b3874f4be6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.715797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.716694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.716822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.717011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.737536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c98711-c7e8-43ac-8274-c6262eff841e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.747581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.783642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:30.784793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:30.785261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:30.786233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:30.798491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.800413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.803727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:30.805780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:30.818032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:30.819047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:30.820000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:30.821413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 377/751] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:30.823893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.891017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:30.930970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 375/752] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:30.932063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:30.956259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade82f6f-037f-4c9b-84c6-3f4eb14c9356 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:30.998462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.005347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.006084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.006523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.006905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.015102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.016976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:31.032894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:31.034021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.035181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:31.036986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 378/753] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:31.038046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.058117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54b3d-95da-4149-971e-f8124342e6ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.104052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.105264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.106088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.106546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.107804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:31.108283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.110340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.133378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 376/754] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:31.135430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.151444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed343506-cf0d-4bfe-8f9f-8cf9b3caa013 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.163139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.189917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.190884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.191266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.191686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.203008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.204573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:31.225021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:31.226145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:31.227300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:31.227300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:31.227300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:31.227300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:31.233157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 379/755] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:31.233157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.256985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1844ce-34b6-4fe4-91e1-9eb1af314906 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.271752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.294575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.295508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.295931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.296359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.297423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:31.297870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.299572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.312826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 377/756] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:31.314290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.338329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c5283-18a7-4a75-a01b-d0e5c8aed8de None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.384965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.386302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.387168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.387624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.403239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.404926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:31.426308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:31.427304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:31.428230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:31.428230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:31.428230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:31.428230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:31.432456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 380/757] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:31.433487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.443991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc7-ab49-48b9-a336-f3369b5db8e5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.450312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.481910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.482958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.483573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.484042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.485207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:31.485805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.487678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.506567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 378/758] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:31.507878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.540764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.542838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e0d2-ab3b-4ed5-8d8d-6de768124f78 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.577937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.578708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.578993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.579267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.580104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01612217d9fb486daccdc549d9e0c6fb) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:31.580417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.581865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.587329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 381/759] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/role_assignments?user.id=01612217d9fb486daccdc549d9e0c6fb&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:31.588098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.638699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c366d-e16e-49cf-ab6b-a18c978930ff None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.679433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.680033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.680319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.680596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.681449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:31.681762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.683315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e5b1a-8e05-462d-b14c-62a4fc8517fb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.697275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 379/760] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:31.698759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45d5e-76a3-4cbe-abf1-8403855b1acc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.699050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45d5e-76a3-4cbe-abf1-8403855b1acc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.699443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45d5e-76a3-4cbe-abf1-8403855b1acc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.705043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 380/761] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:31.706056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45d5e-76a3-4cbe-abf1-8403855b1acc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.718113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.748925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c74741-4655-4738-9aa4-f846fff86e08 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.787126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.787930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.788219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.788500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.795440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.797168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f3c6f1b9-5952-42cf-86f0-7dc25e33ce25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:31.812413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:31.813356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:31.814253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 382/762] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:31.816006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.816360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.816709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.827069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:31.834436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:31.835751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:31.839136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45d5e-76a3-4cbe-abf1-8403855b1acc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:31.876314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:31.877563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:31.878052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:31.878663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:31.886537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:31.888329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:31.903539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:31.904479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:31.905394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:31.908121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 381/763] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:31.909186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:31.930500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 383/764] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:31.930500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.008836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.039533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b69d0d-b626-4f1a-b28f-00d2a68140ca None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.056108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6499a-d603-4bd4-8802-63b3f3fee690 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.077443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.078556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.079068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.079584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.080711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.081227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.082910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.096420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 382/765] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.097666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.108033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.108716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.109006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.109263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.110328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.110546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.111981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.120282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.126488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 384/766] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.127639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.211838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4aa6-f37f-494c-8000-a7467de3b3ac None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.254668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.255563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267cbd-3593-4b45-9d16-abc606f06cd8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.256096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.256577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.257168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.258129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.260050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:32.273940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:32.275050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:32.275890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:32.283941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 383/767] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:32.285033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.286311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.291375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.293274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.293274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.293274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.294146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=6a190fb9da064c5e9d4eea2abb9cdfdf) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.294581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.296093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.302497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 385/768] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/role_assignments?user.id=6a190fb9da064c5e9d4eea2abb9cdfdf&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.307585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.381703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.410345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f1bf-ebba-49e1-ad87-5ba0b3347e06 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.426077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a5a91-4dcb-42e2-87b8-0ea2d70d0a82 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.449575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.450784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.451275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.451761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.455097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=789dd1683eac49d9b26609eb2903475e, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.456860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.463882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.464658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.465017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.465391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.466450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.466690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.468166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683a46b-7eee-438d-98f4-61b016d4bfd5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.475325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 386/769] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.475887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa23cd-a88e-4920-b243-cbe9e9bd59ef None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.476241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa23cd-a88e-4920-b243-cbe9e9bd59ef None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.476582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa23cd-a88e-4920-b243-cbe9e9bd59ef None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.481696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 384/771] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:32 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:32.482166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 387/771] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:32.483636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.484139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.484679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.484911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.502414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:32.503765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:32.571601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.588856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 388/772] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:32.614686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.688259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75926d-e6a3-477a-a226-a48a257482a5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.737707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3762f6-caed-4058-9a96-9cd3bf1144f0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.761884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.762609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.762901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.763761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.764508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.765119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.767164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.772354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.779323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 385/773] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.784702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.793652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.795640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.796082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.796521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] PATH_INFO: `/v3/services/network` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.797402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.799860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:32.820167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:32.821430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:32.822489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:32.824079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 389/774] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:32.830096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.894080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:32.913686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d8e9-3a9a-49cd-9ae0-68999a7c205c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.953887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc78f-3ab1-4cf1-9f8f-64d27611a089 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:32.968676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.969732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.970209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.970696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.971773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] RBAC: Adding query filter params (interface=public, service_id=03146d0498fc4e3cbe100b07aad46ac7, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.972307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.973928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:32.982473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 386/775] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/endpoints?service_id=03146d0498fc4e3cbe100b07aad46ac7&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:32.988125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:32.995957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:32.996680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:32.997016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:32.997292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:32.998052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] RBAC: Adding query filter params (name=network) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:32.998361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:32.999850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:33.005334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 390/776] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:33.015362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:33.135636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:33.147995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0329c-0c45-4df1-ae80-df0413bc6ddd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:33.181718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:33.209372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:33.210050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.210335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.210661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.211390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:33.211858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:33.213247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:33.218940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 391/777] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:33.221155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1b6a1-26b8-4e79-aff7-e185d9b2aa1f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:33.224645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:33.268983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:33.270098 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.270608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.271080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.271983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:33.273548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:33.279664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 387/778] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:32 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:33.372757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6891-0e9f-49ba-8451-eec43b90dfc9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:33.420981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:33.421665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.421978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.422252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.423084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] RBAC: Adding query filter params (interface=public, service_id=a01a7a38d9fa44a790322d17a3397c0f, region_id=RegionOne) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:33.423398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:33.424762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2f27e-0327-4231-83f2-df9c7050b9ea None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:33.433224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 392/779] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:33 2022] GET /identity/v3/endpoints?service_id=a01a7a38d9fa44a790322d17a3397c0f&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:33.439282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:33.583253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90f988-0fff-4e1d-845b-d091a2b0405e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:33.612718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:33.621899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:33.622609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.622899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.623177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.623873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:33.625347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104e5af-cde1-46ff-b1db-441630655fd2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:33.637267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 388/780] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:33 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:33.882720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd45ccc-961f-475a-a82e-4076366ae87b None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.883242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd45ccc-961f-475a-a82e-4076366ae87b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.883242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd45ccc-961f-475a-a82e-4076366ae87b None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.885179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 393/781] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:33.891052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:33.891349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:33.891633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:33.910326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:33.911568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:34.004742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 389/782] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:34.051843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd45ccc-961f-475a-a82e-4076366ae87b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.126117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:34.200398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd45ccc-961f-475a-a82e-4076366ae87b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:34.244883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:34.245696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.245980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.246263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.256971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:34.264996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:34.265965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:34.266851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:34.270502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 394/783] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:34.272725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.393816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e4561-c008-4363-bb70-96acaf620874 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:34.428011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:34.444941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:34.445628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.445915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.446196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.446997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:34.447312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:34.448725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:34.462000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 390/784] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:34.467652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.589879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab04e63-bd3a-4a45-8418-8fd0600a50b3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:34.591506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:34.627020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:34.627930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.628275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.628614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.641256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:34.643005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-35473bab-cf4c-475e-a48f-700ecef39402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:34.667696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:34.668675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:34.669618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:34.669618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:34.669618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:34.669618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:34.675999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 395/785] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:34.677187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.728592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef144-a1c6-4827-8e3b-4747a7a67f49 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.729229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef144-a1c6-4827-8e3b-4747a7a67f49 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.729728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef144-a1c6-4827-8e3b-4747a7a67f49 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.731660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 396/786] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:34.736927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.737461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.737943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.755761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:34.757473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:34.805051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74806d-bf7c-4c1e-9141-b9a71eb732a3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:34.838632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:34.839305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:34.839597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:34.839881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:34.840612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:34.840924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:34.842311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:34.858694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 391/787] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:34.860221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.900900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 397/788] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:34.907460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:34.945814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:34.975294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b38adf-7a50-4230-bb02-ae0a5afc81e4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.020921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.021733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.022076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.022357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.030064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.031672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:35.047730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:35.048733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:35.050121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 392/789] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:35.055643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.072332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.091166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8ac4-caa0-48ec-be42-d53833514e64 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.151639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73205485-639f-436b-9654-278b63c14f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.152830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205485-639f-436b-9654-278b63c14f0f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.153307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205485-639f-436b-9654-278b63c14f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.153798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205485-639f-436b-9654-278b63c14f0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.175616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73205485-639f-436b-9654-278b63c14f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:35.184910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53059862-78d2-43ca-812f-0dfe459dfd77 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.187357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-73205485-639f-436b-9654-278b63c14f0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:35.189149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:35.193244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:35.194140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:35.197105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 398/790] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:35.197105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea849f3-e2aa-4798-9f4e-ff816e2c25c6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.197521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea849f3-e2aa-4798-9f4e-ff816e2c25c6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.198212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea849f3-e2aa-4798-9f4e-ff816e2c25c6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.206799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 399/791] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:35.207605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea849f3-e2aa-4798-9f4e-ff816e2c25c6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.220976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.221661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.221944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.222219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.223099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:35.223411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.224950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77236c7-6856-4e44-af0b-2809017ad383 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.243350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 393/792] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:35.244606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.244898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.245179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.266779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:35.267973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:35.338188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea849f3-e2aa-4798-9f4e-ff816e2c25c6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.354400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 394/793] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:35.354400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.397601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.398619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.399054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.399560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.400523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:35.400979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.402572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.418398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 400/794] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:35.419552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.430150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.444039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.454836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8eb6-7c22-4a76-bc69-8aedae49dce9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.491622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.492474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.492760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.493039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.506001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.508003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:35.533366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:35.534330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:35.535584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:35.535584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:35.535584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:35.535584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:35.536387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e35e8-2711-4f0a-99c3-7fb6c92cedb7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.538456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 395/795] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:35.539550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.584137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.584948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.585293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.585965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.600560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:35.610852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:35.611837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 401/796] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:35.617777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.668029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6caba-093f-459a-be5f-212a87a6c995 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.697641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.710975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.712150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.712440 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.712721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.726060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.727819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.730983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.741461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a22df-a7fd-41e8-aee5-7f5cf31c47d5 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:35.751795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:35.752726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:35.753618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:35.753618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:35.753618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:35.753618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:35.758085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 396/797] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:35.759614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.784843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.785945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.786783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.787266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.788232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:35.788776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.790634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.804622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 402/798] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:35.805777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.883352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920be8-aff0-4146-bb71-228056715f2f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.915067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02ab7-20d8-44d9-9ca3-2d0e52576fcb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:35.919770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.920456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.920743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.921021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.921911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:35.922226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.923635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.937010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 397/799] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:35.937010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:35.950788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:35.961911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:35.962993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:35.963484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:35.963984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:35.964913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:35.965511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:35.967169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:35.980383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 403/800] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:35.981570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.000983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.056571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6f9ab-07a8-4aee-bb9d-a1eb353a8198 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.088441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d57-a5bb-470a-8b65-af17a37caf0a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.092183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.092998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.093289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.093609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.100962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.102432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-dacf3f1b-aa7f-494c-b613-162de9513518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:36.116934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.117921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 398/801] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.119357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836a9f-d44d-4dca-9464-a622b134a0d9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.120028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836a9f-d44d-4dca-9464-a622b134a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.120028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836a9f-d44d-4dca-9464-a622b134a0d9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.121103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 399/802] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:36.121774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836a9f-d44d-4dca-9464-a622b134a0d9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.129885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.130724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.131027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.131702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.143313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.144693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-17cc076a-91e8-47ac-9368-e61a41527062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:36.161696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.162800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.163705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:36.163705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:36.163705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:36.163705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:36.163705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 404/803] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.164162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c213-b107-41a8-9c84-7193a27dca03 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.164366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c213-b107-41a8-9c84-7193a27dca03 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.164641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c213-b107-41a8-9c84-7193a27dca03 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.165887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 405/804] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:36.166587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c213-b107-41a8-9c84-7193a27dca03 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.210507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.226484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836a9f-d44d-4dca-9464-a622b134a0d9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.243429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.261167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.261993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.262277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.262584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.269784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.270750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c213-b107-41a8-9c84-7193a27dca03 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.271264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c573530a-d8ce-41cc-9f34-8e69ce6d697b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:36.287024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.288074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.289003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 400/805] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.289609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1026-fc55-4c51-8ca4-7d93f8bf536b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.290139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1026-fc55-4c51-8ca4-7d93f8bf536b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.290139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1026-fc55-4c51-8ca4-7d93f8bf536b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.306434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-678c1026-fc55-4c51-8ca4-7d93f8bf536b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:36.307196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.307611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-678c1026-fc55-4c51-8ca4-7d93f8bf536b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:36.307864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.308146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.308422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.309215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:36.309499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.310975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.324198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 406/806] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:36.324996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.376290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 401/807] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:36.377577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.377849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.378123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.395435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:36.396552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:36.432596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.436589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42606b1-a49b-4c6a-83d1-8c60faa68d87 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.461773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 402/808] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:36.462630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.470325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.471108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.471456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.471799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.472596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:36.472970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.474416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.490351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 407/809] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:36.490351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.515891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.576170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649ce69-ecdf-42da-be58-797da41d5b65 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.585130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd4d7e-98c6-4fa2-8bdd-600850e3ec6f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.610444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.611091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.611398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.611673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.612511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:36.612818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.614264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.617900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.618813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.619090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.619360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01612217d9fb486daccdc549d9e0c6fb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.621335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=01612217d9fb486daccdc549d9e0c6fb, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.622778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.637967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 403/810] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:36.637967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.638709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 408/811] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:36 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01612217d9fb486daccdc549d9e0c6fb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:36.638709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.669372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.745984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.754837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0887ac-6a92-40d7-9f10-4dc36231551d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.777796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2e853-b6c1-40d1-9b83-19c83d42bea6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.800259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.801335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.801614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.801890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] PATH_INFO: `/v3/services/network` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.802801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.804886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.816674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.817989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.819116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 404/812] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.820096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.820197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.820289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.820289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.820289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.820289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.822875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.838663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.839822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 04:02:36.840823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:36.841936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 409/813] 10.222.0.16 () {56 vars in 1129 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.847025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.929811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57265-708b-48b3-91a0-3ef08f77938b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.937056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:36.956749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000183-7b3d-4dd3-a67a-a0a779b68f0e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:36.967625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.968438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.968777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.969066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:36.977522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:36.979019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:36.992900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:36.994023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:36.995079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:02:36.996076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:36.996608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:36.996676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 405/814] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:36.996676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:36.997194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:36.997591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.010055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.011763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.021573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:37.031793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:37.032671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:37.033542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:37.033542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:37.033542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:37.033542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:37.033542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 410/815] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:37.033988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.111998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860f32-c547-49e2-a31d-612730c20882 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.140007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751723d0-d572-4cee-91d6-6aa70cf0e75d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.149540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.150218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.150532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.150812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.151529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] RBAC: Adding query filter params (name=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.151836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.153205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.158687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 406/816] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.159487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.181649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.182251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.182568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.182847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.183677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.183987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.185350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.186471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.194522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 411/817] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.196027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.246653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.264773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b7e24-98d0-421c-be80-73e62758d06b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.307634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.309093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.309532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.309993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.310680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf46799-1ae8-4bfc-9483-f1def2fb6b22 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.311185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.311648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.314392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.332487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 407/818] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.333310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.346970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.347861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.348323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.348761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.349504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.349819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.351213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.363458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 412/819] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.368599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.429894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.442787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d65caa-cc80-4046-a72b-8cd2abe9f5bb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.473868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46484-36fe-4c52-a5f9-be7a79e75dad None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.477945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.478641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.478924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.479195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.479906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.480213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.481577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.487026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 408/820] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.487832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.497869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.513569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.514244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.514572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.514851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.515797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:37.516108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.517480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.522799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 413/821] 10.222.0.16 () {56 vars in 1124 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:37.523522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.599779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46918a-918b-4182-8612-fcf4357722e6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.633219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a9da-8e32-4860-a1cf-ac1d9e201615 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.634573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.635415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.635647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.635981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.659370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.672051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.672937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.673223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.673500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.680445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.681871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:37.696179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:37.697280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 414/822] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:37.698174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.715857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.717618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.726542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.807425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8908261-4664-4a16-ba32-8c40dd4d961c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.827188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 409/823] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:37.828063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.846567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:37.847583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:37.848010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:37.848541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:37.849435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:37.851178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:37.874406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 415/824] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:37.875193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:37.939109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:37.955229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3ca8-b5d9-465a-b3a3-d3f62bbf8efe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:37.964207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.004913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.005890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.006319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.006770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.007121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12839f0-ab1a-4b53-8f0c-5384384186cb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.007838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.009907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.035401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 410/825] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:38.036502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.076649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.077339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.077632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.077978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.079045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:38.079366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.081563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.102406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 416/826] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:38.107297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.171346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22965af-6d6a-4b09-aae1-fe01f318c031 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.225812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.226614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.229372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.229680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.242705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.244224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.264216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:38.266202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:38.267190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:38.268163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:38.268163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:38.268163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:38.268163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:38.272980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 411/827] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:38.274310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.276508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef01a-a7ed-4ef0-9ce7-64d436cd50ae None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.290121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.319431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.320949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.322357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.322924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/6a190fb9da064c5e9d4eea2abb9cdfdf/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.340314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=6a190fb9da064c5e9d4eea2abb9cdfdf, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.341765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.360207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 417/828] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:38 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/6a190fb9da064c5e9d4eea2abb9cdfdf/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:38.408782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76d544-6a4a-487c-976d-faef0a41b8d2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.448082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.448924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.449219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.449500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.450247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:38.450592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.452095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.470433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 412/829] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:38.471903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.580841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c218-0f8e-4bc6-a854-b702951819df None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.616713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.617524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.617809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.618087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.636267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.637757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-daa4bc5c-26a6-4a05-b12d-2680721f7385 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:38.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:38.654059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:38.655212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:38.658631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 418/830] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:38.660450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.672014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845622f5-71ab-4c59-af4a-c25f549bbc6e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.672390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845622f5-71ab-4c59-af4a-c25f549bbc6e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.672779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845622f5-71ab-4c59-af4a-c25f549bbc6e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.674417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 419/831] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:38.679526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.679809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.680083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.691728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.696749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:38.697961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:38.754730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.774456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 420/832] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:38.782525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb9663-41a2-4599-b8ce-13c3d99ca6eb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.794508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.825451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.826128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.826600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.826887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.828287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:38.828610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:38.830234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:38.846539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 413/833] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:38.854339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:38.904914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df3af-07f6-49c3-83fb-6c72ceb13dd5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.952758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:38.954225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:38.954598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:38.955136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:38.957762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:38.969516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:38.977343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:38.978268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:38.979532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 421/834] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:38.985541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f266e-3f8b-4ac7-aff4-42174abb15ef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:38.986631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:39.024618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:39.033661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.034702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.035345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.036277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.037552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:39.038040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.040302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.048511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 414/835] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:39.100732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9df5-25cc-4f48-bc31-dc39b8e17116 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:39.140754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.141441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.141727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.142007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.142815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:39.143128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.144535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.154974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 422/836] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:39.160836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:39.271944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c252-59ac-435e-ba8e-86f3cd0e1374 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:39.309143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.309948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.310233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.310543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.321814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.323301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:39.340796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:39.341710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:39.342596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:39.342596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:39.342596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:39.342596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:39.342596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 415/837] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:39.349055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:39.408923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:39.456136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56dcb6-589a-4d99-a540-2da148a4d54d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:39.490907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.491941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.492433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.492899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.493807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:39.494305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.496209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.627627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 423/838] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:39.633484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:39.684808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:39.740392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d3b01-96e3-4a3d-84d5-a6a71fa9713b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:39.779422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.780235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.780527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.780807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.788883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.790330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:39.804736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:39.805691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:39.806662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 416/839] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:02:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:39.813478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:39.916182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e18dcb-e0aa-4da6-b847-21b04b88b6a6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:39.933379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:39.951302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:39.951914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:39.952198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:39.952498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:39.953479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:39.954022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:39.955241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc15b7-256d-4cd0-b1a9-d63189a4b011 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:39.972979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 424/840] 10.222.0.16 () {56 vars in 1135 bytes} [Mon Jul 4 04:02:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:39.978572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.089273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be7cb4-7486-4203-8b19-b6e7accd804b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.106709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.124421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.125290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.125579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.125863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.197097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.198575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.217282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9cfa-feb6-43aa-bd30-0891b7060591 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.217470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9cfa-feb6-43aa-bd30-0891b7060591 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.217753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b9cfa-feb6-43aa-bd30-0891b7060591 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.219475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 425/841] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:40.224506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.224787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.225058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.241498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:40.242702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:40.316422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 417/842] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:40.318855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.328773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.329659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 426/843] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:40.351393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.434112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad12f1e-fcf9-486b-a98b-4a86f3411c8e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.453791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e16e-1b4d-49b2-86bf-aaafff433343 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.456133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.472798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.473911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.474430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.474901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.486442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.488174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.491654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.492464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.492745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.493018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.493679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.495061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:40.507170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:40.508146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:40.508957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:40.511151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:40.511975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:40.512850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:40.512850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:40.512850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:40.512850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:40.513716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 427/844] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:40.514608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.515499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 418/845] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:40.524096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.586976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.621020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95478223-9d23-47ae-b3fc-a9598bf7a759 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.630585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814cba1-a4c5-41e4-b0cf-f50ad2a124ac None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.663850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.664523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.664806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.665086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.665782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:40.666097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.666717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.667499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc0444-023b-4b27-bbbc-c70d06254b4c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.667683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.668115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.668683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.669833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:40.670568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.672121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6adcde-55d5-42b8-950d-fb6a9276cf5d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.672340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 428/846] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:40.673566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94748126-11ea-4374-9a91-0af242571922 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.673852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94748126-11ea-4374-9a91-0af242571922 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.674130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94748126-11ea-4374-9a91-0af242571922 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.679247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 429/847] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:40.680013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94748126-11ea-4374-9a91-0af242571922 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.693816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 419/848] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:40.695174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.695469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.695751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.711928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:40.713099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:40.718281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.781546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 420/849] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:40.782400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.810114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94748126-11ea-4374-9a91-0af242571922 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.833702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:40.850942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.851714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.851995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.852267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.852980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:40.853285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.854887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.866521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 430/850] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:40.866521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:40.897232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9962f5-1832-4532-bb03-5f6632b57f6e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:40.941523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:40.942488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:40.942770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:40.943177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:40.951356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:40.952914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:40.969575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:40.970611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:40.971555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 421/851] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:40.976603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.016226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.026861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8897e7-63c8-413e-8fa7-7d6cf59dec23 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.068457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.069622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.070110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.070634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.085446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:41.093729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9684-1995-4f82-91e6-55c5a536d170 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:41.093886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:41.094911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:41.095757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 431/852] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:41.100977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.107998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.141200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189924d-f927-4894-b282-0081de66e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.142241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189924d-f927-4894-b282-0081de66e272 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.142810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189924d-f927-4894-b282-0081de66e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.143281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189924d-f927-4894-b282-0081de66e272 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.144295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189924d-f927-4894-b282-0081de66e272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:41.144821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189924d-f927-4894-b282-0081de66e272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.146935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189924d-f927-4894-b282-0081de66e272 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.163651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 422/853] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:41.170022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189924d-f927-4894-b282-0081de66e272 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.223746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63123b-ec5b-4c6c-9b5b-d7281cb5b1e3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.264128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.264799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.265088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.265483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.266318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:41.266614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.267931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0f0a-86b8-4b5a-8f65-27607686d862 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.277076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 432/854] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:41.278292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1889-3f5b-49e0-a730-01f95a61309d None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.278606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1889-3f5b-49e0-a730-01f95a61309d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.278882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1889-3f5b-49e0-a730-01f95a61309d None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.280239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 433/855] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:41.282786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce1889-3f5b-49e0-a730-01f95a61309d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.292713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189924d-f927-4894-b282-0081de66e272 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.307780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.402022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce1889-3f5b-49e0-a730-01f95a61309d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.437774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.439062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.439347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.439627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.449472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.450190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.450473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.450696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.451211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.451513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01612217d9fb486daccdc549d9e0c6fb) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:41.451820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.452639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.453309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f66a3-f629-4315-932b-9d06750b75d1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.459728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 423/856] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/role_assignments?user.id=01612217d9fb486daccdc549d9e0c6fb&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:41.460668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.460950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.461221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.478835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:41.480079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:41.480962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:41.481813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:41.482752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:41.482752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:41.482752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:41.482752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:41.489880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 434/857] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:41.491285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.524080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.582399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 424/858] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:41.598510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.601789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.642216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829fb81-0644-4d8e-a2cd-f1e6a15e897f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.686452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.687032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.687823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.688106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.689339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:41.689635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.691047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.708269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 435/859] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:41.709844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.733139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1ba20-7b6c-4e39-8ba6-40f689a69b68 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.778078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.779064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.779349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.779626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.780288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.782094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:41.799126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:41.800097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:41.800957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 425/860] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:41.807315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.820295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa56068-9a67-4b38-aa99-cbfbf17fea94 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.845191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.852721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:41.860060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.861031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.861411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.861783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.869525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.871110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:41.888122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:41.889627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:41.891836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:41.893089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 436/861] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:41.899512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:41.940878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ebe83-bdab-44e3-99a3-967220c349d0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:41.983481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:41.984457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:41.985017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:41.985466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:41.986745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:41.987458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:41.989520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:41.995904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 426/862] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:42.001962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:42.013085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823cf7-0b1e-4cb7-953b-c472bd3150c7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.054554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.055267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.055516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.055794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.057058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:42.057372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.059119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.083908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 437/863] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:42.091359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:42.106831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:42.121904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61090551-6dba-4fb9-8ae7-71f1bbcb606f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.150059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:42.175504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.176276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.176630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.176975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.177760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:42.178160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.181185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.190445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 427/864] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:42.214196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08f9a-a68a-4245-8704-5335aec859a1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.252109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.253079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.253547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.253959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.328322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.330039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.381517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:42.435236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 438/865] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:42.440969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:42.547149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dbcf5-46b0-4894-b5a6-b35a3181d372 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.582956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.583915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.584200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.584485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.595336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.596792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:42.615766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:42.616710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:42.617588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:42.617588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:42.617588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:42.617588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:42.621618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 428/866] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:42.629590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:42.739446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6206b-9f75-46f1-bcd1-9216b882f988 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.776795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.777464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.777749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.778032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.778799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:42.779107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.780475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc076f2-3e24-4b15-be60-097146c7a7ff None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.797765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 439/867] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:42.799206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:42.808372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:42.901372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d708fe6-4894-497e-8921-fcccafa34e88 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:42.934167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dd27d-3069-41d0-9e06-4b1982260532 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.934502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dd27d-3069-41d0-9e06-4b1982260532 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.934768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dd27d-3069-41d0-9e06-4b1982260532 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.936724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 440/868] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:42.938622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:42.939428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.939707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.939981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.945658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:42.945936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:42.946210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:42.950221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:42.953235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:42.963525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:42.964698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:42.966826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:42.972303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:42.973202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:42.974137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:42.980635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 429/869] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:42.982426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:43.046549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 441/870] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:43.061249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:43.066170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:43.106615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3454439-085b-45ae-ac58-3c03261f4827 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:43.142928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:43.143651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.143930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.144204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.145043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:43.145351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:43.146897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:43.164079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 430/871] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:43.166246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:43.198490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518de0f-0803-4ba7-9c30-96d7a45695a2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:43.234024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:43.234951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.235350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.235722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.236673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:43.238017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5e4d7-9d04-487f-904a-4dff0aa12821 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:43.246562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:43.258835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 442/872] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:43 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:43.297457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f112f3-18d2-4d0a-82be-08140c56604d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:43.328206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:43.345191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:43.346226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.346722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.347180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.348251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=6a190fb9da064c5e9d4eea2abb9cdfdf) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:43.348769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:43.350454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaba160-c2af-41a3-9d5c-f644172e1bef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:43.362526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 431/873] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:43 2022] GET /identity/v3/role_assignments?user.id=6a190fb9da064c5e9d4eea2abb9cdfdf&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:43.637109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682962-34cb-4b6f-b6a2-50122a888bc0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.637401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682962-34cb-4b6f-b6a2-50122a888bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.637683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682962-34cb-4b6f-b6a2-50122a888bc0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.639346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 443/874] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:43.644641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.645116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.645194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.662073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:43.663279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:43.729873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 432/875] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:43.752541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4682962-34cb-4b6f-b6a2-50122a888bc0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:43.857211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4682962-34cb-4b6f-b6a2-50122a888bc0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:43.891879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:43.892614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:43.892894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:43.893168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:43.893834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:43.895228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:43.907645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:43.908654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:43.909658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 444/876] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:43.914999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:44.017989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e97ea-0d49-4d6f-b0ab-73b04c026758 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:44.035916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:44.054112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:44.054869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:44.055161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:44.055444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:44.056169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:44.056522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:44.057975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedc046-c8e6-4e06-bbf5-3b59c343056e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:44.063933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 433/877] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:44.069137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:44.139930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:44.179596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4946c44-f0d3-439a-9f50-986e5bebca70 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:44.213719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:44.214716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:44.215196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:44.215652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:44.216529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:44.217005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:44.218788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca078-25d7-4b53-9c6b-26a3df6424e6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:44.224289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 445/878] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:44.376212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28644ce-4e18-4b14-9c56-76650788cb66 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:44.376502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28644ce-4e18-4b14-9c56-76650788cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:44.376777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28644ce-4e18-4b14-9c56-76650788cb66 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:44.378929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 434/879] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:44.384894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b5dc-8a75-4282-b9f3-22ea7c60bf6d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:44.385188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b5dc-8a75-4282-b9f3-22ea7c60bf6d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:44.385473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b5dc-8a75-4282-b9f3-22ea7c60bf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:44.403343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0b11b5dc-8a75-4282-b9f3-22ea7c60bf6d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:44.404536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0b11b5dc-8a75-4282-b9f3-22ea7c60bf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:44.479210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 446/880] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:44.502232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28644ce-4e18-4b14-9c56-76650788cb66 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:44.625200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28644ce-4e18-4b14-9c56-76650788cb66 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:44.660618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:44.661385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:44.661581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:44.661866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:44.662619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:44.663986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9eac3f-5cea-4a4b-a628-488e0782373e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:44.680424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 435/881] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:44 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:45.384917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429016-9606-4615-8718-14071cbb1035 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.385777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429016-9606-4615-8718-14071cbb1035 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.386192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429016-9606-4615-8718-14071cbb1035 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.387955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 447/882] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:45.393536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f43-d57c-4705-a009-d50b20886400 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.394089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f43-d57c-4705-a009-d50b20886400 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.394630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f43-d57c-4705-a009-d50b20886400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.412101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-57697f43-d57c-4705-a009-d50b20886400 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:45.413591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-57697f43-d57c-4705-a009-d50b20886400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:45.449775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:45.498426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 436/883] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:45.520949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db429016-9606-4615-8718-14071cbb1035 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:45.578657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:45.581805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ea9d8-d4c8-4ba5-85db-b8b888357ad2 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.582454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ea9d8-d4c8-4ba5-85db-b8b888357ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.582947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ea9d8-d4c8-4ba5-85db-b8b888357ad2 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.586526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 437/884] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:45.591567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.592074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.592687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.613132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:45.615053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:45.630245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db429016-9606-4615-8718-14071cbb1035 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:45.678702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:45.680366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.680833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.681256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.682063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:45.684113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:45.698649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 438/885] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:45.703507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:45.704488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:45.705318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:45.708078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 448/886] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:45.713977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:45.724509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:45.823132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:45.828178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4af67-b582-418c-8443-fd430939742c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:45.831430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fe21e-7a11-4d78-944f-216bf7e86fb4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:45.849228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:45.864242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:45.865416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.865892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.866361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.870691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:45.871367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:45.871659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:45.871941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:45.872656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:45.873013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:45.874348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:45.878891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:45.884691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 439/887] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:45.890611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:45.891549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:45.892440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:45.894390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:45.896606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 449/888] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:02:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:45.898159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.017454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f0c72-a42d-4231-bb50-fe822d6e9151 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.032880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f2cf2-c58e-45db-ab6f-ceba8abc69ad None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.052868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.053635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.053993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.054404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.055165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:46.055547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.056988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.066989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 440/889] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:46.067985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.088624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.089732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.090208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.090698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.091659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:46.092251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.093869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.096218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.103780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 450/890] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:02:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:46.109714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.163116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.195900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2175f3-7eff-4134-bb5e-ec85799facb2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.238130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a603254-5e78-4523-82dd-1dfbabe80cfa None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.245094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.246621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.247118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.248039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.249493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] RBAC: Adding query filter params (interface=public, service_id=7a82a4677f714c44926d889c6db97aec, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:46.250081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.252223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.260122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 441/891] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/endpoints?service_id=7a82a4677f714c44926d889c6db97aec&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:46.265483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.282571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.283391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.283662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.283942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.298980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.300750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-27d45f99-6054-4da2-872a-bd343cb6cc25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:46.319147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:46.320084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 451/892] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529faa-ecda-49ff-b1ac-837a95c788bf None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529faa-ecda-49ff-b1ac-837a95c788bf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529faa-ecda-49ff-b1ac-837a95c788bf None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.325836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 452/893] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:46.328300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea529faa-ecda-49ff-b1ac-837a95c788bf None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.373247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.386894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c7d2-b5d1-4486-bbef-eba4ef3eedfb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.423452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.424038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.424317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.424591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.425273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.426658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9474fc-6295-40d4-ae04-5c14b28932cb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.432463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 442/894] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:02:46.433726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.434097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.434473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.444116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.452464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:46.453834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea529faa-ecda-49ff-b1ac-837a95c788bf None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.453969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:46.500515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.501446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.501797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.502169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.503031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:46.503410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.504977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.518938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 453/895] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:46.524075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.550616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 443/896] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:46.573219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.654035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d9c6-3028-46cf-86dd-39af76dd2e65 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.671810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.693639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.694397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.694698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.695027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.697279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db010b0-12d3-409b-988b-f950889a6334 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.703328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.704761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1c6cc792-6158-4ac3-be73-bc79dd38d850 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:46.721765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:46.722786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:46.723718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:46.727672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 454/897] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:46.728987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a16d4-4d3f-42bc-8a1d-40006a48b31f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.729278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a16d4-4d3f-42bc-8a1d-40006a48b31f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.729565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a16d4-4d3f-42bc-8a1d-40006a48b31f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.731154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 455/898] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:46.731856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a16d4-4d3f-42bc-8a1d-40006a48b31f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.740950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.742218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.742564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.742841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.743580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.745198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb4b0-3f6a-4334-8a46-d7b79332f9a2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.749266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:46.761464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 444/899] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:46 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:02:46.762757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.763051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.763333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.781662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:46.782818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:46.874395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 445/900] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:46.876188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a16d4-4d3f-42bc-8a1d-40006a48b31f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:46.891678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.923470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:46.924167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:46.924446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:46.924718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:46.925572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:46.925878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:46.927334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:46.947878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 456/901] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:46.954593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:46.977986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.016161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab79bc-3fbe-42d9-b7d7-f591f99c0698 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.043664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.054339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.055516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.055949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.056466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.057280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.058985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:47.072765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:47.073928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:47.074962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:47.077530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 446/902] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:47.080475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8e0c8-c0ed-4b8c-93f8-41e30a5c889d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.083045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.132980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.133644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.133926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.134578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.202840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829477da-a214-4cf3-a858-4a7eda8c47ed None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.217343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.219109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.241721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.248876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.249979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.250265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.250641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.251638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:47.251949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.254002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.261260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 447/903] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:47.267563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.296976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.334940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 457/904] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:47.340727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.393307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41402a9-6e26-425a-8da3-020776235349 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.428008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.429036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.429479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.429917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.430818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:47.431273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.433018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.439258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 448/905] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:47.444463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.468662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5c61-13bb-4901-a357-04433a85e66b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.516875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.517706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.517986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.518258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.523625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.533264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.535341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:47.554168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:47.555140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:47.556048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:47.556048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:47.556048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:47.556048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:47.558334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 458/906] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:47.563556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.575352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869fde5-04f2-494c-a38c-a6009d562676 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.610199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.610905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.611240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.611443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.612227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] RBAC: Adding query filter params (interface=public, service_id=3f2da1947c174ac9b886db43d4ba0d47, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:47.613250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.613946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.619782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 449/907] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/endpoints?service_id=3f2da1947c174ac9b886db43d4ba0d47&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:47.626894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.631672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.704858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe04-c5af-4c79-b378-5a7f2cd7fe6c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.743365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.744348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.744836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.745312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.746257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:47.746790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.748369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.754000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae124-e9c7-49b3-85dc-48b1b8f4bd82 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.761704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 459/908] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:47.768651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:47.785202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.798754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.799868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.800363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.800855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.801842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.803570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.810730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 450/909] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:02:47.864721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:47.889017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501d5b-9460-4392-aa98-af7ca07a96da None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:47.927716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:47.928943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:47.929437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:47.929921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:47.938670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:47.940538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:47.956787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:47.957768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:47.958698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:47.961409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 460/910] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:47.967151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:48.074831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468613-3fe0-46a0-a34f-c64a03391256 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:48.109199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:48.109869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.110154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.110474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.111272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:48.111588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:48.113018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64b20c-58b0-4c6a-9e92-5d0f1e825457 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:48.125922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 451/911] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:48.131924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:48.235177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5de840-d1a8-4aa2-beba-e36d3ec485a3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:48.271474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:48.272150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.272440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.272724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.273564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] RBAC: Adding query filter params (role.id=2a19083cee00428688cca4bdf57d9b4f, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=6a190fb9da064c5e9d4eea2abb9cdfdf) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:48.273886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:48.282806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7a346-10ae-4745-b831-4f0f14e55327 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:48.289272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 461/912] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:48 2022] GET /identity/v3/role_assignments?user.id=6a190fb9da064c5e9d4eea2abb9cdfdf&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=2a19083cee00428688cca4bdf57d9b4f => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:48.548378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2163-b920-4dd8-b9a7-f240ee6b974a None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.548788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2163-b920-4dd8-b9a7-f240ee6b974a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.548910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2163-b920-4dd8-b9a7-f240ee6b974a None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.550563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 452/913] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:48.555800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2432ff7-c5e1-4635-8704-ebc70aaf9cad None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.556250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2432ff7-c5e1-4635-8704-ebc70aaf9cad None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.556327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2432ff7-c5e1-4635-8704-ebc70aaf9cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.562343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:48.573568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e2432ff7-c5e1-4635-8704-ebc70aaf9cad None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:48.574775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e2432ff7-c5e1-4635-8704-ebc70aaf9cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:48.656501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 462/914] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:48.679208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2163-b920-4dd8-b9a7-f240ee6b974a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:48.734222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:48.788226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2163-b920-4dd8-b9a7-f240ee6b974a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:48.828588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:48.829331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.829618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.829900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.830605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:48.831997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:48.844424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:48.845527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:48.846448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 453/915] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:48.849353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584792f-f6c8-422b-9a77-d3dddd70cfcf None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.849641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584792f-f6c8-422b-9a77-d3dddd70cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.849919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584792f-f6c8-422b-9a77-d3dddd70cfcf None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.851530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 463/916] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:48.854724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:48.855986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:48.856272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:48.856550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:48.873513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:48.874881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:48.945646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 464/917] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:48.968615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:48.979769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6989c55-ce22-4eb8-9760-d27cbd1e6c2e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:48.998019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.028458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.030119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.030775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.031350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.032499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:49.033525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.033849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.036093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.043955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 454/918] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:49.050041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.078351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54feec2-8702-4f28-83ec-1780bd1d4a63 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.113610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.114336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.114659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.114935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.115589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.116952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:49.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:49.137840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:49.138684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:49.143241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 465/919] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:49.144575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.184276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788277c-791e-4eef-91bd-41ca9e977ea4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.221555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.222336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.222754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.223043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.223760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:49.224075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.225598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.231215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 455/920] 10.222.0.16 () {56 vars in 1114 bytes} [Mon Jul 4 04:02:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:49.236649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.258348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce157e-6e22-4e96-9107-12fc2ae4c832 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.293220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.293653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.306181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.306887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.307175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.307458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.308178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:49.308488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.310421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.318453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 466/921] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:49.324318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.369375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fe638-5330-4a03-a2c5-51f3c7916549 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.411861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.412465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.412750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.413028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.413804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.415265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.443096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 456/922] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:02:49.459973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b32d7-f92a-450e-9063-7a36418332b2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.504160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.505242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.505733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.506199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.507133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:49.507633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.509523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.516290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 467/923] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:49.522177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.638101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.645356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32e1f-42ea-42d3-86a8-cd692c2994d2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.682753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.683490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.683772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.684047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.684832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] RBAC: Adding query filter params (interface=public, service_id=e8adafd3963c40be87c237e5b7d7e4ed, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:49.685143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.686529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c2ab8-3c22-45e1-ae77-f93a6f0af033 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.692037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 457/924] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:49 2022] GET /identity/v3/endpoints?service_id=e8adafd3963c40be87c237e5b7d7e4ed&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:49.697405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:49.753613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:49.802974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8777e84-5aae-4edc-a57f-87ed19137953 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:49.837570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:49.838247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:49.838570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:49.838855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:49.839546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:49.840919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2a6f6-672a-4d4e-adb1-2a8bef3178a3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:49.847802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 468/925] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:49 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:02:50.013398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da082a-2af6-4304-84bb-8857d598fb37 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.013934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da082a-2af6-4304-84bb-8857d598fb37 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.013934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da082a-2af6-4304-84bb-8857d598fb37 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.015961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 458/926] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:50.021319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.021612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.021895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.038981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:50.040163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:50.107271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 469/927] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:50.129859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da082a-2af6-4304-84bb-8857d598fb37 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.233456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da082a-2af6-4304-84bb-8857d598fb37 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.268056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.268841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.269167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.269443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.270101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:50.271509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-4b2b86ed-4338-4587-913a-c4b5c73e979e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:50.284217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:50.285215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:50.287226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 459/928] 10.222.0.16 () {56 vars in 1132 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:50.291683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.346766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:50.384654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599790f-05f7-406a-ace6-5d1a8f629f48 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.385273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599790f-05f7-406a-ace6-5d1a8f629f48 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.385750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599790f-05f7-406a-ace6-5d1a8f629f48 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.388666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 460/929] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:50.393017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.393542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.394011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.403602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb739661-51f9-4ea9-be69-fbc3fb1596dc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.424083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:50.425293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:50.446562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.447041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:50.448076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.448555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.449023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.450110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:50.450772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:50.453133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:50.461481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 470/930] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:50.466355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.510490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 461/931] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:50.528189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.588302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955660f-2b70-4e27-b02e-a10b6c6b65b0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.600469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:50.642961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.643893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.644177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.644453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.645599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:50.645914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:50.647587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:50.648453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d22b36-7e07-482a-b73f-d14fd2c28070 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.654694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 471/932] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:50.661485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.687924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.689148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.689663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.690168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.701127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:50.709520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:50.710643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:50.711474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:50.716581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 462/933] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:50.717838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.739208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:50.776564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ef10-bf11-4996-81d7-9b10e77d310d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.814782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.815607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.815991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.816451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.817448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:50.819083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:50.831241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304a3c-25ee-498d-a669-7663c6a37e1b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:50.842768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 472/934] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:02:50.885807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:50.887138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:50.887713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:50.888190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:50.889223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:50.889737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:50.891790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:50.906537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 463/935] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:50.909670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:50.948228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.035515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750a44-d10a-4f54-886f-67e26a1df0ce None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.071528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.072337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.072621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.072897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.084865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.086303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:51.104209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:51.105181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:51.106767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:51.106767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:51.106767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:51.106767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:51.106767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 473/936] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:51.111280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.184267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.214577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a0fa-6624-42ee-9d96-5d8fee1524be None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.248194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.248866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.249153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.249526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.250251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:51.250603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.252073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.264686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 464/937] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:51.270396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.361189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.380324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f6fae-0a52-4aaa-92eb-0299737264b3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.416456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.417280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.417627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.417888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.425717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.427172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:51.442115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:51.443222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:51.444828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 474/938] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:51.449325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.550336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb924-4d4a-4090-bae9-b37bccb4e169 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.580841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.585899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.586615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.586897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.587198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.588119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:51.588843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.590316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1436862-d7dd-48df-9f3a-0e9aecf6109c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.608834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 465/939] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:51.614396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.685741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30eea-c493-4dda-b6ba-794868884ca0 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.686025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30eea-c493-4dda-b6ba-794868884ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.686302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30eea-c493-4dda-b6ba-794868884ca0 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.688314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 466/940] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:51.693312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.693612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.693889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.710165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:51.711383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:51.727123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4282ab-554d-4667-a1c9-ba33612cdb21 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.769209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.770336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.770908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.771197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.782948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.784387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.785143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 467/941] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:51.801477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:51.801650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:51.802525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:51.803803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:51.803803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:51.803803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:51.803803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:51.803803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 475/942] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:51.806689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.808475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.905144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466346d-bc79-427d-8ddc-d65fb881e1d4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.911022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde863e-872b-467a-b51b-8a60a23919b4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:51.926906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:51.942592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.943403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.943801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.944170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.944970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.946607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.948254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING keystone.server.flask.common [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 04:02:51.949897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:51.950779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.951212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.951719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:51.952934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:51.953386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:51.954927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:51.972459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.sql.core [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 04:02:51.972459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 04:02:51.972459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [parameters: {'id': 'c3867cf863e14ac7ae9530bd31c40380', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 04:02:51.972459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 04:02:51.973227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 476/943] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:51.974273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-5740fd75-ea41-44bc-b2d9-13e8a8c998eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c3867cf863e14ac7ae9530bd31c40380.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c3867cf863e14ac7ae9530bd31c40380. Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:51.989368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 04:02:51.990232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application yield resource Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise self.value Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application session.commit() Jul 04 04:02:51.991206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.flush() Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application compat.raise_( Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 04:02:51.992103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 04:02:51.993466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:51.994404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [parameters: {'id': 'c3867cf863e14ac7ae9530bd31c40380', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.995487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:51.996549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c3867cf863e14ac7ae9530bd31c40380. Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 468/944] 10.222.0.16 () {60 vars in 1140 bytes} [Mon Jul 4 04:02:51 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf61-4f95-4c2a-a8bc-a2c99c4ed8b1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:51.997652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf61-4f95-4c2a-a8bc-a2c99c4ed8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:51.998475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf61-4f95-4c2a-a8bc-a2c99c4ed8b1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.004797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 469/945] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:52.005591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9bf61-4f95-4c2a-a8bc-a2c99c4ed8b1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.044341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:52.096297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e99b7-0efe-4aaf-bede-80f7654b1bf4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.115448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9bf61-4f95-4c2a-a8bc-a2c99c4ed8b1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.135872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.136670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.136953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.137546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.144835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.146280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.151618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.152468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.152749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.153020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.160065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1be4fd3c-5127-4f4e-8982-6176c9cf7ff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:52.160352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:52.161194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:52.162097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:52.163257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.166425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 477/946] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:52.167674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.167960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.168348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:52.176574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:52.177449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 470/947] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:52.184189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.189249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:52.190454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:52.220051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:52.262431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 478/948] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:52.262431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.294024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e47be-21a1-4453-bb06-3444482fb6f8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.309075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:52.328919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.329590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.329873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.330149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.331378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:52.331378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.332695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.349625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 471/949] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:52.349625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.372004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c88d-aa34-4799-9afe-e7cfd79f4310 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.407739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.408495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.408786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.409066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.409898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:52.410279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.411687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.428765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 479/950] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:52.429601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.461945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:52.468651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcd-45d1-45f2-b422-50cea97bf769 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.517778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.518955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.519396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.519810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.520619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.522890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:52.540558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:52.541802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:52.542749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:52.546588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 472/951] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:52.550437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.558737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd63aa-432b-4867-9c1e-16e44fe74e09 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.562562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:52.617500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.618438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.618789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.619154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/6a190fb9da064c5e9d4eea2abb9cdfdf/roles/2a19083cee00428688cca4bdf57d9b4f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.622071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=6a190fb9da064c5e9d4eea2abb9cdfdf, role_id=2a19083cee00428688cca4bdf57d9b4f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.623677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.645097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 480/952] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:02:52 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/6a190fb9da064c5e9d4eea2abb9cdfdf/roles/2a19083cee00428688cca4bdf57d9b4f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:02:52.704710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636f2f-8768-4d91-9432-2092d5110d3c None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.746168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.747218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.747725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.748192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.749151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:52.749707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.751434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.761043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 473/953] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:52.764855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.876355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321955c-2dd5-4ea4-8781-4e74b33967d0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:52.913073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:52.913797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:52.914081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:52.914353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:52.915115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:52.915422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:52.916787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa9c67-ada6-40fa-bd71-5357fb2d34b1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:52.922443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 481/954] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:52.931645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:52.961101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:53.042673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c60c-6f30-48dc-ae43-21be641cc687 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:53.067858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa41b-0634-421c-90e5-baa276aa9f97 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.068178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa41b-0634-421c-90e5-baa276aa9f97 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.068426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa41b-0634-421c-90e5-baa276aa9f97 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.069958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 482/955] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:02:53.076205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.076496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.076774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.083079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a829-4019-411f-9430-767e4016293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:53.083883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a829-4019-411f-9430-767e4016293a None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.084270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a829-4019-411f-9430-767e4016293a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.084645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1a829-4019-411f-9430-767e4016293a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.085508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1a829-4019-411f-9430-767e4016293a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:53.087030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1a829-4019-411f-9430-767e4016293a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:53.095415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:53.097425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:53.110665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 474/956] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:02:53.129868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:53.188584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 483/957] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:53.211872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a829-4019-411f-9430-767e4016293a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:53.308924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:53.331382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1a829-4019-411f-9430-767e4016293a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:53.366302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:53.367138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.367423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.367702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.368364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:53.369733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:53.383496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:53.385183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:53.386276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:53.390025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 475/958] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:53.391474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:53.491190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f3c98-eac1-4987-ac66-723b43f378b6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:53.526294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:53.527226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.527649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.528065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.529013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:53.529458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:53.531182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe39659-961b-4127-aa46-d49cd30f9564 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:53.536532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 484/959] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:53.546390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:53.646548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64235-9185-4840-8b4d-87dadb94365e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:53.680983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:53.681655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:53.681936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:53.682208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:53.682951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:53.683374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:53.684753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b97b5-c6c2-4601-a868-81a0bdd29221 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:53.690046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 476/960] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:54.658359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19260c47-ed71-4e41-ab21-5961a894a502 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:54.658917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19260c47-ed71-4e41-ab21-5961a894a502 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:54.658917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19260c47-ed71-4e41-ab21-5961a894a502 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:54.660553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 485/961] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:54.665744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:54.666169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:54.666343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:54.683778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:54.685290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:54.687646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:54.756278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 477/962] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:54.777303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444c0d-e226-4a2d-b435-5cbfeb82086f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:54.777587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444c0d-e226-4a2d-b435-5cbfeb82086f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:54.777868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444c0d-e226-4a2d-b435-5cbfeb82086f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:54.779521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 486/963] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:54.781783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:54.784598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:54.784878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:54.785153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:54.802461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:54.803692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:54.819750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:54.886546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 487/964] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:54.894765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cfacc-0a00-4cca-b2e0-a344643c621d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:54.902912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:54.935807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:54.936445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:54.936756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:54.938601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:54.938601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:54.941243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:54.950872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:54.953469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.sql.core [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 04:02:54.953469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 04:02:54.953469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [parameters: {'user_id': '0c5f2e904aca4c22b99b60ee0a85dcf7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 04:02:54.953469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:54.962010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:54.963784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application yield resource Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application next(self.gen) Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise self.value Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application session.commit() Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 04:02:54.965604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.session.flush() Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application compat.raise_( Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 04:02:54.966874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 04:02:54.968138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application conn.query(q) Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application result.read() Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 04:02:54.976969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [parameters: {'user_id': '0c5f2e904aca4c22b99b60ee0a85dcf7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:54.978694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 04:02:54.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 478/965] 10.222.0.16 () {60 vars in 1131 bytes} [Mon Jul 4 04:02:54 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:54.982204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.005678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1544c-541e-423d-b297-ca323deab63b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.040750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.041437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.041720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.041993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.052950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:02:55.061521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:55.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:02:55.065421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:55.074603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 488/966] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:02:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:55.075870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.093629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.093812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281799cb-7b02-4ad8-8abd-f42573be7565 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.140541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.141546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.141903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.142250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.152331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.154865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:55.174011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:55.175066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:55.176002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 479/967] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:02:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:55.180800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.215113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1439858-51e0-4f8e-bdc9-f0d24d203c40 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.245532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.260277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.260949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.261228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.261501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.262683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:55.263049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.264757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585f8a-c279-43d5-b48d-bad48baae402 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.280607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 489/968] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:55.280607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acefda2-2f36-4d4d-b316-c248c45aba95 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.280607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acefda2-2f36-4d4d-b316-c248c45aba95 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.280607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acefda2-2f36-4d4d-b316-c248c45aba95 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.280607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 490/969] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:55.281004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acefda2-2f36-4d4d-b316-c248c45aba95 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.288522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4aa8-33ee-4a2b-82f9-4bf1b11e7183 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.326089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.327229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.327229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.327627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.328290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:55.328607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.330048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268466-4e26-41ad-bb0a-bea70f42ea4e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.364989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.369623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 480/970] 10.222.0.16 () {56 vars in 1101 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:55.373063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.373681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.374191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.399822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:55.402708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:55.404951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acefda2-2f36-4d4d-b316-c248c45aba95 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.447940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.448841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.449127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.449445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.466405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.468606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:55.493444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:55.494443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:55.495349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:55.495349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:55.495349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:55.495349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:55.495658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 481/971] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:55.497918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 491/972] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:55.504102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.522670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.570541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.632243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0024dc-170e-44e2-9774-778220b3ed06 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.660223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e761d-71b5-4b29-80e6-f8c55707867e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.676867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.677544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.677826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.678099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.678869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:55.679174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.680544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.691920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.693576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 482/973] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:55.699856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.705253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.706141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.706529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.706871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.707599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.709164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:55.722997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:55.724245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:55.725281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 492/974] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:55.730661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.801507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79abe19-329c-4c37-9255-c83cd0d88d5b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.810548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.833814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792b33-4358-446a-827d-bcfe2ff03006 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.835974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.836798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.837086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.837363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.845174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.846769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-0db190d3-692d-4e39-865e-5c889f0028c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:02:55.861522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:55.862670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:55.863918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 483/975] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:55.864688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fa1f-29b1-4896-a564-43fd2579d5de None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.864688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fa1f-29b1-4896-a564-43fd2579d5de None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.864870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851fa1f-29b1-4896-a564-43fd2579d5de None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.866274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 484/976] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:55.868739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851fa1f-29b1-4896-a564-43fd2579d5de None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:55.871354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:55.872028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.872305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.872614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.873292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:55.873597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:55.875034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b4aaa-cd7d-4608-bb7e-9e2d204aec28 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:55.880047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 493/977] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:55.883737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:55.884335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:55.884632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:55.902299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:55.903540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:55.949659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:55.971764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 494/978] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:55.972149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851fa1f-29b1-4896-a564-43fd2579d5de None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:55.972563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.006170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.006886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.007160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.007437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.008276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:56.008586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.010331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.027300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 485/979] 10.222.0.16 () {56 vars in 1149 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:56.028058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.054562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.076653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ecd2d-4937-4bff-8f69-034b88c09ecb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.110711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.111457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.111729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.112000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.112707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:56.113010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.114397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.124425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 495/980] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:56.124425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.130576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f0d68-563b-42da-b8d2-89c8d6b71827 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.164434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.164592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.165116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.165398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.165669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.166444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.167857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.181257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 486/981] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:02:56 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:56.182114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.236610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe748-ff8d-481d-93a7-24ef73f593a7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.275044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.284479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.285688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.285973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.286251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.294582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ec2d-a467-4ed0-a316-46164e6cbb68 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.333824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.334041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.334158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.334470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.335278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] RBAC: Adding query filter params (interface=admin, service_id=e8adafd3963c40be87c237e5b7d7e4ed, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:56.335595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.339916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.345794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 487/982] 10.222.0.16 () {56 vars in 1249 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/endpoints?service_id=e8adafd3963c40be87c237e5b7d7e4ed&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:56.359081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.376306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.377985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.425119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.479209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee6620-3f3f-43be-9f10-fff366ebd9d5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.489075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 496/983] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:02:56.494513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.520338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.521118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.521399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.521784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.522493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.523885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.530779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 488/984] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:02:56.599040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431899a-7887-41d3-b802-018405e67e6f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.660550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.662166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.662456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.662766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.673904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.677154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.678689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:02:56.698889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:56.700152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:56.701646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:02:56.701646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:02:56.701646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:02:56.701646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:56.701646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 497/985] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:56.707658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.725017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:56.812365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e0dfc-7757-4b73-aa8d-8b084a2d20d2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:56.846624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:56.847378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:56.847726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:56.848061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:56.848844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:56.849216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:56.850668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:56.863248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 489/986] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:56.868719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:56.972399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43005a9-4252-44d5-8661-9faef5e94b34 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:57.008753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:57.009868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:57.010148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:57.010443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:57.018690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:57.020126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:02:57.033836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:57.034800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:02:57.035717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:57.038709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 498/987] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:57.040985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:57.122640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:57.149575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc8ee-1ff1-4e63-b08e-79a433147a84 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:57.185575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:57.186402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:57.186868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:57.187240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:57.188135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:57.188553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:57.190077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf85096-1d0f-475f-bbef-973cb877311e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:57.207245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 490/988] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:02:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:57.209112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:57.265929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:57.317238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639dcce-6b2b-469c-b08a-b79187cbf18c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:57.354971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:57.355622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:57.355898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:57.356171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:57.356996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] RBAC: Adding query filter params (role.id=2a19083cee00428688cca4bdf57d9b4f, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=6a190fb9da064c5e9d4eea2abb9cdfdf) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:57.357307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:57.358754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91af3-7a56-4623-abe7-3e4d50391d18 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:57.364805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 499/989] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:02:57 2022] GET /identity/v3/role_assignments?user.id=6a190fb9da064c5e9d4eea2abb9cdfdf&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=2a19083cee00428688cca4bdf57d9b4f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:58.235844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f3a5-e3e2-4a4b-94fe-5fb78fe7ef92 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.236424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f3a5-e3e2-4a4b-94fe-5fb78fe7ef92 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.236424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f3a5-e3e2-4a4b-94fe-5fb78fe7ef92 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.237961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 491/990] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:58.242921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.243339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.243507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.260684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:58.261869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:58.330915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 500/991] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:58.355456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3f3a5-e3e2-4a4b-94fe-5fb78fe7ef92 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.462575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3f3a5-e3e2-4a4b-94fe-5fb78fe7ef92 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:58.504547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:58.505619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.505915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.506189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.506880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:58.508253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:58.510701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-61c7dd9f-60e8-41d0-a273-bb3854e780c9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:58.522353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:58.523393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:02:58.524228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.525775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 492/992] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:02:58.529508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.576583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:58.625891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7339-9c0d-4cf8-abff-84e32af5c22d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.626287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7339-9c0d-4cf8-abff-84e32af5c22d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.626498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7339-9c0d-4cf8-abff-84e32af5c22d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.627911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 493/993] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:58.633064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009153-8478-44ab-903b-91dc80085443 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.634765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009153-8478-44ab-903b-91dc80085443 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.635058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009153-8478-44ab-903b-91dc80085443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.641611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450f597-7980-4921-abc6-4f81d6b24656 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:58.653291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-21009153-8478-44ab-903b-91dc80085443 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:58.654621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21009153-8478-44ab-903b-91dc80085443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:58.680156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:58.680824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.681123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.681386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.682108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:58.682445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:58.683820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:58.693853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 501/994] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:02:58.695017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.728303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 494/995] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:02:58.753169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009153-8478-44ab-903b-91dc80085443 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.802816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a748-7519-463a-a41e-3122c9a837c4 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:58.814211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:58.839024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:58.839702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.839991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.840269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.840985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:58.841293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:58.842695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:58.848488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 502/996] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:58.854014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.858000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009153-8478-44ab-903b-91dc80085443 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:58.887094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:58.896761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:58.897636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:58.897917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:58.898209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:58.898910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:58.900291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:58.912634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:58.913628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:02:58.914484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 495/997] 10.222.0.16 () {56 vars in 1126 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:58.919944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:58.965269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aefaa1-17eb-44f0-8d22-cfd67122ff24 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.005094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.005814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.006101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.006467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.007276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] RBAC: Adding query filter params (interface=public, service_id=0d44de61239f41078e574eb3a05cafdf, region_id=RegionOne) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:59.007594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.008966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.015076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 503/998] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/endpoints?service_id=0d44de61239f41078e574eb3a05cafdf&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:59.021260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.028151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe27c-fba0-4347-9206-e405f47dbf71 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.060539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:59.068312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.068988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.069282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.069564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.070280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:59.070766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.071993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.081654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 496/999] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:59.083472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.124655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37f703-2641-4447-bce9-62b40a3f56c8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.143845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:59.160888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.161558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.161843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.162119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.162836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.164194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.178412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 504/1000] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:02:59 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:02:59.199792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4812-05c7-47fd-b95b-47fba1c67415 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.239239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.239920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.240211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.240479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.241202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:59.241517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.242956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd20b2-cbf0-49e3-b2fb-5114ea9ed651 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.249038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 497/1001] 10.222.0.16 () {56 vars in 1122 bytes} [Mon Jul 4 04:02:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:59.254741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.363990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3fac9-81d3-4cdd-9040-f26c557a5ec8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.365499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:59.393228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83bb4-28c7-49e3-8957-a218dd2d8267 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.393555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83bb4-28c7-49e3-8957-a218dd2d8267 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.393801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83bb4-28c7-49e3-8957-a218dd2d8267 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.395216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 498/1002] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:02:59.400295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.400648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.400929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.407223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.407896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.408173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.408448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.409200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.410611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.418403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:02:59.419671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:02:59.427392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 505/1003] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:02:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:02:59.505354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 499/1004] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:02:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:02:59.528481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.638080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec22bcc-0ff4-4f73-a43a-276e75c72ddd None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.672468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.673268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.673549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.673824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.674517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.675878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:02:59.687506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:02:59.688724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:02:59.689737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 506/1005] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Jul 4 04:02:59 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:02:59.694697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.742174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:59.799468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c506-9013-4310-86cf-9faa6647afb3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.833776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.834464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.834743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.835017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.835726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:59.836027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.837391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069383a-f58e-47fe-999b-d75f65c880e5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:02:59.846436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 500/1006] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:02:59.848956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:02:59.878526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:02:59.960684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f9a4e-cd43-4c2c-839c-dc37a313bcd8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:02:59.995343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:02:59.996013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:02:59.996303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:02:59.996582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:02:59.997297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] RBAC: Adding query filter params (type=image) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:02:59.997612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:02:59.999019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f07ad6-2432-424a-b7f7-66395019b79c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:00.010452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 507/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:02:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:01.221053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f88dc-8015-400b-8f54-0a4242db4e7c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.221645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f88dc-8015-400b-8f54-0a4242db4e7c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.221645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f88dc-8015-400b-8f54-0a4242db4e7c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.223166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 501/1008] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:01.228156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eee6c7-48ac-4d6a-8154-765f9bf3db05 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.228571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eee6c7-48ac-4d6a-8154-765f9bf3db05 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.228740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eee6c7-48ac-4d6a-8154-765f9bf3db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.245870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-28eee6c7-48ac-4d6a-8154-765f9bf3db05 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:01.247075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-28eee6c7-48ac-4d6a-8154-765f9bf3db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:01.317313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 508/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:01.339824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f88dc-8015-400b-8f54-0a4242db4e7c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.442909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f88dc-8015-400b-8f54-0a4242db4e7c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.453871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:01.456629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726878-a5ba-4e2a-ad1c-931d39d43647 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.456950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726878-a5ba-4e2a-ad1c-931d39d43647 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.457256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726878-a5ba-4e2a-ad1c-931d39d43647 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.458865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 509/1010] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:01.466675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.466962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.467241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.482739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.483588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.483889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.484163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.484689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:01.484849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.485858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:01.486177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:01.497848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:01.498908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 04:03:01.499881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:01.504071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 502/1011] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:03:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:01.504587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:01.504763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.565794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 510/1012] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:01.589880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.610030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d1243-ac23-4f6e-bd28-eb52569961d1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.649708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.650405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.650698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.651362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.652085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:01.652539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.653909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:01.659416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 503/1013] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:03:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:01.664755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.703989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f555cd-0d87-417e-88f7-4dd662da68e4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.706193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:01.739254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.739981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.740299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.740569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.741742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.743196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:01.744796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 511/1014] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:01.750498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.769412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:01.774992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a133f7-bd6e-429b-9e6d-33f288a33f89 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.810764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411282a-8712-4f23-a456-605745462a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.811886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411282a-8712-4f23-a456-605745462a33 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.812320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411282a-8712-4f23-a456-605745462a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.812600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411282a-8712-4f23-a456-605745462a33 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.813308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411282a-8712-4f23-a456-605745462a33 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:01.813612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411282a-8712-4f23-a456-605745462a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.815003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411282a-8712-4f23-a456-605745462a33 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:01.824411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 504/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Mon Jul 4 04:03:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:01.825611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411282a-8712-4f23-a456-605745462a33 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:01.857571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7aee7-cb5a-4d58-80ef-aacdeafd8af8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.895103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.895775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.896064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.896371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.897093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.898568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87daf8-f4d4-47a0-8279-69c64fe1c711 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:01.924029 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 512/1016] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:03:01 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:03:01.942465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411282a-8712-4f23-a456-605745462a33 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:01.944520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:01.983295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:01.984044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:01.984325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:01.984602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:01.985356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:01.986762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c5ebb-6ab8-47ce-aac5-4dd422112763 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:02.014410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 505/1017] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:03:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:03:02.028105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c20d5-6157-4af6-a37d-b0d9a386da55 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:02.028434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c20d5-6157-4af6-a37d-b0d9a386da55 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:02.028674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c20d5-6157-4af6-a37d-b0d9a386da55 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:02.030151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 513/1018] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:02.036477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311f26-eacf-4068-8a94-82fb4bae8dcc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:02.036800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311f26-eacf-4068-8a94-82fb4bae8dcc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:02.037027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36311f26-eacf-4068-8a94-82fb4bae8dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:02.055781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-36311f26-eacf-4068-8a94-82fb4bae8dcc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:02.057005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-36311f26-eacf-4068-8a94-82fb4bae8dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:02.148125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 506/1019] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:02.170927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c20d5-6157-4af6-a37d-b0d9a386da55 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:02.246810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:02.285545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c20d5-6157-4af6-a37d-b0d9a386da55 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:02.321397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:02.322070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:02.322453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:02.322691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:02.323454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:02.324819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058219d9-240a-45c6-9ee7-93d609210a0b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:02.340425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 514/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:03:02 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:04.006710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430908be-aaa2-48ea-9f42-f633e6f84d95 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.007224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430908be-aaa2-48ea-9f42-f633e6f84d95 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.007325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430908be-aaa2-48ea-9f42-f633e6f84d95 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.008673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 507/1021] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:04.014227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629deb-4327-4fe4-ae36-c787b8f41bb2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.014789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629deb-4327-4fe4-ae36-c787b8f41bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.014965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629deb-4327-4fe4-ae36-c787b8f41bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.032233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-18629deb-4327-4fe4-ae36-c787b8f41bb2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:04.033466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-18629deb-4327-4fe4-ae36-c787b8f41bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:04.111503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 515/1022] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:04.136209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430908be-aaa2-48ea-9f42-f633e6f84d95 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.199929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a2f-5186-4115-a080-4efb97079cca None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.200264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a2f-5186-4115-a080-4efb97079cca None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.200467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a2f-5186-4115-a080-4efb97079cca None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.201828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 516/1023] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:03:04.206978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.207238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.207516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.224065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:04.225231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:04.227632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.247704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430908be-aaa2-48ea-9f42-f633e6f84d95 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.279413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.286711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.287530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.287819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.288097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.289273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.290751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.292536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 508/1024] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:04.298256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.300080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 517/1025] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:04.323131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.407951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c0fc5-0cbc-4ecb-b7b9-f2d3540e4b68 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.425901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81c88-b68a-4e25-b0fd-c148bfef394a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.441913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.442627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.442910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.443187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.443893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.445314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a256a9-4345-45ad-9b76-9393a5fb9abb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.459569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.460389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.460672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.460950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.466095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 509/1026] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:03:04 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:03:04.467800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558e7e5-3660-4d2d-b970-7cc041182d83 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.468467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558e7e5-3660-4d2d-b970-7cc041182d83 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.468467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558e7e5-3660-4d2d-b970-7cc041182d83 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.469680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 510/1027] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:04.475629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.475930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.476234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.480919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:04.489778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:04.490910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:04.491752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 518/1028] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:04.497341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.497628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:04.498984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:04.501331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.553824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.585752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 511/1029] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:04.610773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.623214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828286c1-33b2-4d2d-a695-94e64365b936 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.668959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.669638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.669921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.670220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.671173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:04.671496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.672872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.688378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 519/1030] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:04.690045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.726048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07148de9-1805-4478-823c-29a8103f3b9d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.760128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.760923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.761202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.761475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.762129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.763526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:04.775405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:04.776676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:04.777714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 512/1031] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:04.782422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.785189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.798344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee11c-d494-4537-8e90-83ce6362d5bb None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.807980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:04.834847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.835650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.835926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.836197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.847836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.849274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:04.866582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:04.867604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:04.868481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:04.868481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:04.868481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:04.868481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:04.868481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 520/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:04.873284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.891289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c882-1868-4df1-9f31-c1a9ac5d3dd8 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:04.929012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:04.929344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:04.929625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:04.929896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:04.930640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:04.930948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:04.932310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:04.941813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 513/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:04.942858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:04.984806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20defe5f-3c3d-49db-b3ef-b9767dfc3948 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.015800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.021119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.021794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.022068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.022412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.023190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:05.023512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.024870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.041490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 521/1034] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:05.042632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.055461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed530e3-b312-4991-87aa-de208d67a324 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.065705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.091224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.091897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.092180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.092458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.093434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] RBAC: Adding query filter params (type=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:05.093754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.095149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.099959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 514/1035] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:05.105827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.148594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0204c-6028-407a-8813-bd1357c56e8d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.185080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.186250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.186759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.187279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.195829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.197545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.207516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9adb-a640-499f-9f85-6ea4c4f62988 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:03:05.212597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:05.213595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:05.215231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 522/1036] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:05.219836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.240995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.248206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.248903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.249198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.249599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.250247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] RBAC: Adding query filter params (interface=public, service_id=ca8fdaf92743468b9065ec6f0efa4327, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:05.250584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.251947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.257227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 515/1037] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/endpoints?service_id=ca8fdaf92743468b9065ec6f0efa4327&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:05.262981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.305695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.327653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05222db-171e-43f3-91e1-a28f3e9c7df8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.364237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.364926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.365490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.365772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.366637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:05.366947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.368376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.368770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a729b-5dac-49b3-a58f-139ac25777ed None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.386216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 523/1038] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:05.392162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.412150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.412935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.413217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.413491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.414178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.415714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.431918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 516/1039] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:03:05.496027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b69473-5397-4edc-b604-8d53e911f329 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.534880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.535552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.535827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.536101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.545397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.615794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.617161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.726248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 524/1040] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 04:03:05.731827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.801793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:05.847562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde247e8-02fd-48b2-9f61-4a78e6ef82e2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:05.881601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:05.882321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:05.882643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:05.882920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:05.894170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:05.895655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:05.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:05.914328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:05.915523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:05.915523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:05.915523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:05.915523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:05.918725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 517/1041] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:05.920323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:05.961211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:06.024625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa1ab4-0875-4480-8f36-6874c1c9c08b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:06.060708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:06.061441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.061845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.062146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.062886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:06.063197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:06.064555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:06.076464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 525/1042] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:06.082409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:06.196143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecb50a-ceda-4e43-9b27-3f668a52468e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:06.206895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:06.232111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:06.233163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.233721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.234147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.241340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:06.243085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:03:06.257661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:06.258734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:06.260161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:06.262259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 518/1043] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:06.267909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:06.369250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:06.373578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f2c9-e0d0-49e8-a761-77093a50f5c2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:06.414900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:06.415551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.415837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.416113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.417002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:06.417312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:06.418846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4988e35-3ade-45b5-82cf-83a28b510d6c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:06.437431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 526/1044] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:06.439258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:06.552111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad690-b8bb-42f0-a591-4bf62e613294 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:06.553638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc41935-eb4e-4eb3-8dfc-1fef273c7556 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.553928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc41935-eb4e-4eb3-8dfc-1fef273c7556 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.554205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc41935-eb4e-4eb3-8dfc-1fef273c7556 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.563003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 527/1045] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:06.568394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.568691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.569064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.587152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:06.588351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:06.590000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:06.590695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.590978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.591271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.592087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] RBAC: Adding query filter params (role.id=2a19083cee00428688cca4bdf57d9b4f, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:06.592397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:06.593873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:06.606393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 519/1046] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:06 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=2a19083cee00428688cca4bdf57d9b4f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:06.675926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 528/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:06.699642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:06.739836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:06.828622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc8647-732a-43f0-b177-070594de4aba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:06.864017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:06.864813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:06.865092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:06.865364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:06.876064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:06.883654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:06.884872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:06.885732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 520/1048] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:06.891197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:06.981279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:06.995182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e054b3e-240d-47d3-95ad-6f04e2994fca None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.030753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.031525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.031710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.031976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.032726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:07.033035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.034395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.043207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 529/1049] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:07.048850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.144585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:07.152924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05783d-02c9-4d86-b7b3-ef073090ff37 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.186972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.187855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.188129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.188405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] PATH_INFO: `/v3/users/00646cafff2a4f8595732969d07e7c0a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.201719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] RBAC: Authorizing `identity:get_user(user_id=00646cafff2a4f8595732969d07e7c0a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.203612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61773209-6de0-476f-ba97-e3dc95045a9b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.205508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 521/1050] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/users/00646cafff2a4f8595732969d07e7c0a => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:07.211145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.317546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211094e1-86ec-4ccf-95bb-2e292648828d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.351937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.352590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.352869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.353142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.371930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:07.422214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.423606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.446395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42a6e-f074-47bc-a8e3-d1532a2b8210 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.446725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42a6e-f074-47bc-a8e3-d1532a2b8210 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.447010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42a6e-f074-47bc-a8e3-d1532a2b8210 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.448392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 522/1051] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:03:07.453164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.453449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.453734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.470401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:07.471610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:07.537073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 530/1052] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:07.542697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.550413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 523/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:07.568344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.599464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:07.656548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:07.663213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32458bc7-bb81-424f-bf9d-35f4ba00519b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.677052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c312817-5e7b-40d6-aa0d-c852c70976c9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.700132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.701020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.701306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.701588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.702987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.704465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.706458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 531/1054] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:07.712668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.719114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.719926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.720212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.720488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.721219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.723077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:07.735345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:07.736342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:07.737167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 524/1055] 10.222.0.16 () {56 vars in 1105 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:07.742515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:07.817480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee18e1-592a-4f4c-8038-dc4570819807 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.834838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:07.851892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.852556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.852838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.853118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.853949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=3d5dec4686d84b9183d6154c5c515418, user.id=00646cafff2a4f8595732969d07e7c0a) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:07.854253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.855736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.857160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84df0b2-1dbc-4bdf-b128-98d553786ca5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:07.861441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 532/1056] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/role_assignments?user.id=00646cafff2a4f8595732969d07e7c0a&scope.project.id=3d5dec4686d84b9183d6154c5c515418&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:07.895857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:07.896544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:07.896827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:07.897096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:07.897811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] RBAC: Adding query filter params (name=image) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:07.898115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:07.899536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:07.910786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 525/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:07.911829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:08.026605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aef456-476b-4309-8a6d-73579b021931 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:08.062227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:08.062931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:08.063190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:08.063496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:08.064204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] RBAC: Adding query filter params (type=image) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:08.064516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:08.065887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e10c63-86b4-4256-8117-a900b8e33111 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:08.071153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 533/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:08.076832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:08.091592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:08.185873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9fad-c146-4162-ab0b-2c9faba1d019 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:08.221601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:08.222270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:08.222590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:08.222861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:08.223593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:08.224950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3533d-ff5f-48d1-983e-48eb06f89417 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:08.244629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 526/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:03:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:08.639641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37581c21-898f-4302-89f3-159649dbf3f2 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:08.639924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37581c21-898f-4302-89f3-159649dbf3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:08.640201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37581c21-898f-4302-89f3-159649dbf3f2 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:08.641765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 534/1060] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:08.647018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:08.647310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:08.647588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:08.665350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:08.666550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:08.728847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:08.744882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 527/1061] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:08.768503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37581c21-898f-4302-89f3-159649dbf3f2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:08.815954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:08.872685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37581c21-898f-4302-89f3-159649dbf3f2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:08.913806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:08.914572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:08.914851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:08.915122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:08.925645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:08.933733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:08.934879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:08.935772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 535/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:03:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:08.943712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.045357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f13d3e-2c5f-4644-a8d4-0d99fafda9c2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.079209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.079874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.080157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.080428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.081176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:09.081477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.082885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.091232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 528/1063] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:03:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:09.097043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.199149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52a8-1668-4ba0-8724-a96154a44f8d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.205986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:09.235486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.236276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.236563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.236820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.246874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.248309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:09.266616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:09.267537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:09.268429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:09.268429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:09.268429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:09.268429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:09.268429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 536/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:09.273532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.330230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:09.377978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90633-41ba-4b0f-beb6-9a1570c8ca40 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.415224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.415887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.416165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.416436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.417147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:09.417449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.418851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.431473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 529/1065] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:09.437527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.541196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463828-9260-4020-be2c-858a3abd3c93 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.577319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.578114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.578386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.578695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.586199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.587668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:03:09.601977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:09.602999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:09.604064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:09.606480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 537/1066] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:09.609514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.720827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a690ae0-96d0-4355-b541-d60c58d03fb0 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.735800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:09.758830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.759813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.760092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.760362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.761197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:09.761508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.762964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73237-d036-4885-88a9-322f896470d1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.781799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 530/1067] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:09.787516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.807434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:09.890194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779b87-1257-4be0-bf50-1d5df4d8ea88 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.890620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779b87-1257-4be0-bf50-1d5df4d8ea88 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.890775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779b87-1257-4be0-bf50-1d5df4d8ea88 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.891495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ef0dc-4990-472e-84cc-9886da891cb2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:09.892294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 531/1068] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:03:09.897216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.897512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.897794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.915177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:09.916359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:09.927588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:09.928407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:09.928675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:09.928965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:09.940230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:09.941859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:09.960677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:09.961610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:09.962566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:09.962566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:09.962566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:09.962566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:09.962566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 538/1069] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:09.967984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:09.990891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 532/1070] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:10.013696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.049624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.087332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354be6-a101-4eee-92cd-97334d7f19ac None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.117441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d69a3-5738-4e6f-97e2-9b36833a4a51 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.119293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.123581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.124312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.124590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.124824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.125551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:10.125854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.127253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.144493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 539/1071] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:10.145539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.160584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.161390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.161670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.161936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.172376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:10.180175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:10.181094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:10.181969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 533/1072] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:10.187726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.253203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d773eb0-ae0a-42bb-9fda-035af2145839 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.282305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.290664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.291577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.291991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.292281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.296271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b3a2-2805-46e1-a8c7-cbe5e4dd1ce9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.300882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.302314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3d28b140-26a7-47c7-a65c-35d1ea9a56cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:03:10.316498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:10.317450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:10.318358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 540/1073] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:10.319683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39af7-84ca-4f88-bb70-04d157741aa9 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.319683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39af7-84ca-4f88-bb70-04d157741aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.319683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39af7-84ca-4f88-bb70-04d157741aa9 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.321036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 541/1074] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:10.323320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39af7-84ca-4f88-bb70-04d157741aa9 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.333940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.334651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.334943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.335224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.335987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:10.336298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.337747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93270e-211d-4a5e-9628-b781a65b527a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.346327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 534/1075] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:10.347421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.347716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.347996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.364043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:10.365235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:10.382752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.427728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39af7-84ca-4f88-bb70-04d157741aa9 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.437491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 535/1076] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:10.438319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.467264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.468359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.468745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.469033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.469839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:10.470157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.471650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.484035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 542/1077] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:10.484838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.538038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.545556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a998e-8f45-4eed-8949-580416577900 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.578678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.579551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.579909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.580210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] PATH_INFO: `/v3/users/00646cafff2a4f8595732969d07e7c0a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.581416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] RBAC: Authorizing `identity:get_user(user_id=00646cafff2a4f8595732969d07e7c0a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.582913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.584598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 536/1078] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/users/00646cafff2a4f8595732969d07e7c0a => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:10.585318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.588395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfb731-1adf-4dff-b5d0-45785e79492e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.610323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.627354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49190828-da26-40a8-abe9-070600485df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.627354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49190828-da26-40a8-abe9-070600485df7 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.627354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49190828-da26-40a8-abe9-070600485df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.627354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49190828-da26-40a8-abe9-070600485df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.687344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1525f-399b-4c70-ba26-f716f0f0f52f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.697407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49190828-da26-40a8-abe9-070600485df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.698915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49190828-da26-40a8-abe9-070600485df7 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.726664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.727556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.727818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.728085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/2a19083cee00428688cca4bdf57d9b4f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.730277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=789dd1683eac49d9b26609eb2903475e, role_id=2a19083cee00428688cca4bdf57d9b4f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.731727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.750008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.761420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 537/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:03:10 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/789dd1683eac49d9b26609eb2903475e/roles/2a19083cee00428688cca4bdf57d9b4f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:03:10.761420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.810715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 543/1080] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:10.817138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49190828-da26-40a8-abe9-070600485df7 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.865797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:10.889082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262e7b-fc38-463e-a611-716a77f0f5d5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.932075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.933350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.933463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.933695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.934889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.936032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49190828-da26-40a8-abe9-070600485df7 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:10.936508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:10.938453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 538/1081] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:10.944294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:10.974502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:10.975315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:10.975601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:10.975880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:10.986838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:10.988283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:11.006840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:11.007769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:11.011308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:11.011308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:11.011308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:11.011308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:11.011308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 544/1082] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:11.015378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:11.018914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:11.055529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dabb8-25b3-4714-9d58-53eb252a2d25 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:11.091919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:11.093179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:11.093494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:11.093782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418/users/00646cafff2a4f8595732969d07e7c0a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:11.095858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d5dec4686d84b9183d6154c5c515418, user_id=00646cafff2a4f8595732969d07e7c0a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:11.097312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38be0f-99ae-457a-845b-78e6aad67226 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:11.113975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 539/1083] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:03:10 2022] PUT /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418/users/00646cafff2a4f8595732969d07e7c0a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:03:11.124431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fb49f-9867-4118-a8ff-119ae91efdf8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:11.164023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:11.164741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:11.164999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:11.165311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:11.166287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:11.166651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:11.168051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824090-9116-4ffe-9217-c85f5217c729 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:11.181907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 545/1084] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:12.755025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d7d3-f7fd-4e0e-a701-bede11d721ee None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:12.755576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d7d3-f7fd-4e0e-a701-bede11d721ee None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:12.755576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d7d3-f7fd-4e0e-a701-bede11d721ee None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:12.756948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 540/1085] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:12.762493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:12.762960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:12.763052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:12.781276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:12.783262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:12.843617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:12.856321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 546/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:12.880783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d7d3-f7fd-4e0e-a701-bede11d721ee None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:12.894400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:12.997016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d7d3-f7fd-4e0e-a701-bede11d721ee None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.032898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.033719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.034005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.034279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.045001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-aa8d67f6-b46b-409b-a631-2953084d5eed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:13.053247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:13.054304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:13.055180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 541/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Mon Jul 4 04:03:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:13.060591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.157428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081efbb-09d7-4191-a1b0-3b9375419f32 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.157622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081efbb-09d7-4191-a1b0-3b9375419f32 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.157878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081efbb-09d7-4191-a1b0-3b9375419f32 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.159448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 542/1088] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:03:13.163951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9dd88-ccd5-41a7-bb59-c31af2fc6c7e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.164341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3202d-45a6-4199-882c-a809b973760b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.164633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3202d-45a6-4199-882c-a809b973760b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.164915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3202d-45a6-4199-882c-a809b973760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.182458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-09f3202d-45a6-4199-882c-a809b973760b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:13.183691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-09f3202d-45a6-4199-882c-a809b973760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:13.198332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.199066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.199343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.199613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.200361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:13.200666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.202211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.214608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 547/1089] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:13.215716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.248636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.254065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 543/1090] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:13.272198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.277843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3202d-45a6-4199-882c-a809b973760b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.321958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d087a-65b5-48ea-ad32-5d22a66ea425 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.354681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.355481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.355767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.356045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.367393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.368831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.378342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3202d-45a6-4199-882c-a809b973760b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:13.386859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:13.388036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:13.389234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:13.389234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:13.389234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:13.389234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:13.389234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 548/1091] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:13.393880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.415406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.416137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.416424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.416709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.427045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:13.434664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:13.435842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:13.436831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 544/1092] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:03:13.441824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.494213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8dd1-e901-4ef7-8509-2b597827b17a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.499726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.501109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.529730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.530419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.530708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.530994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.531713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:13.532016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.533381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.541980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee621-3fb1-4098-b077-c17213b36286 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.544583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 549/1093] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:13.550384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.582250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.582945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.583295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.583569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.584317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:13.584623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.585987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.595543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 545/1094] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:13.601220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.652008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ab18-2c98-4702-8f00-b102e8b2a5f8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.684553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.685267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.685553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.685824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.693179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.694627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.702588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3983-f09d-485f-b9a4-9f6f8a46fa5e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:03:13.709040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:13.709977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:13.711283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 550/1095] 10.222.0.16 () {56 vars in 1093 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:13.715896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.726583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.738992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.739872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.740150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.740422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] PATH_INFO: `/v3/users/00646cafff2a4f8595732969d07e7c0a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.741603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] RBAC: Authorizing `identity:get_user(user_id=00646cafff2a4f8595732969d07e7c0a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.743102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.743881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.749836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 546/1096] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/users/00646cafff2a4f8595732969d07e7c0a => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:13.750645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.816066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e81533-586a-4f78-8fb0-0cb44b7128a6 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.848951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.849615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.849900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.850176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.851041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:13.851349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.852774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.855011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3feb1-ca35-436e-b0b2-db5dd1caeae1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:13.870205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 551/1097] 10.222.0.16 () {56 vars in 1133 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:13.876001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:13.890540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:13.891197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:13.891475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:13.891747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:13.948162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.958397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:13.959034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:13.959845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:13.976144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36baac1d-3b3d-4f7b-ba6e-ad5a182bb896 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.010900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.011644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.011923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.012191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.059212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 547/1098] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:14.065378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.079024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.081313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.156008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.166151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7227f-cb43-4ec6-9c57-190e895732bd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.166567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.183611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 552/1099] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 04:03:14.189403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.204893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.205693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.205975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.206254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.207520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.208980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.210691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 548/1100] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:14.216624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.287041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1fa81-f2e0-44ee-a323-a38d8fb80c4d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.317547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783126-c871-427a-a917-82c0f2cefb0f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.320720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.321519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.321802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.322077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.332530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.333963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:14.351172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:14.352154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:14.353308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:03:14.353308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:14.353308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:03:14.353308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:14.353683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.353784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 553/1101] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:14.353865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.353865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.353865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.353865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=3d5dec4686d84b9183d6154c5c515418, user.id=00646cafff2a4f8595732969d07e7c0a) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:14.354161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.355508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.358548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.361646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 549/1102] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/role_assignments?user.id=00646cafff2a4f8595732969d07e7c0a&scope.project.id=3d5dec4686d84b9183d6154c5c515418&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:14.401972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.473901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3212-59bf-4163-b295-7a7fda4e8b05 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.512051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.512757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.513045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.513390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.514149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:14.514542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.516103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.527764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 554/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:14.535481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.563283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.651516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd698419-7d27-4d1a-b082-d7d4bc82db57 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.685923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.687129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.687412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.687691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.694534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.695965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:03:14.710552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:14.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:14.712530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 550/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:14.718125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.818870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.825613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e05369-b2b1-414f-b70e-0a3013496318 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:14.862580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:14.863244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:14.863526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:14.863800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:14.864568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:14.864872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:14.866313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c517ce2-daa6-430c-bd40-2c8fe3e7e65a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:14.879591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 555/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:14.885765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:14.973617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:14.993244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97b4fa-be3f-42ca-b328-e24dce2e6708 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:15.028972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:15.029635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:15.030109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:15.030401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:15.031268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] RBAC: Adding query filter params (role.id=2a19083cee00428688cca4bdf57d9b4f, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=789dd1683eac49d9b26609eb2903475e) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:15.031580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:15.033016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f11ee-f14f-48aa-9b05-53a0edb95bba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:15.039320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 551/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:14 2022] GET /identity/v3/role_assignments?user.id=789dd1683eac49d9b26609eb2903475e&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=2a19083cee00428688cca4bdf57d9b4f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:16.475314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b67c8-731d-4c9d-9136-a7953090d55c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:16.476137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b67c8-731d-4c9d-9136-a7953090d55c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:16.476137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b67c8-731d-4c9d-9136-a7953090d55c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:16.478313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 556/1107] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:16.484093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:16.484519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:16.484608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:16.502470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:16.503647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:16.583029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 552/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:16.608167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b67c8-731d-4c9d-9136-a7953090d55c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:16.724792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b67c8-731d-4c9d-9136-a7953090d55c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:16.761829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:16.762633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:16.762909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:16.763181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:16.764550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:16.765972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f1c80-8733-4ae3-b8cf-411969fbf003 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:16.767606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 557/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:16.773887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:16.859860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:16.885865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24911b-c749-495c-bea4-572a70eeb237 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:16.921460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:16.922155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:16.922464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:16.922745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:16.923464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:16.924935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e526b0-aafb-426e-9c35-d3f762798038 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:16.947825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 553/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:03:16 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:18.972424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b22f-8cda-429c-8ff3-5111c093fae5 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:18.972954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b22f-8cda-429c-8ff3-5111c093fae5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:18.972954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b22f-8cda-429c-8ff3-5111c093fae5 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:18.974228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 558/1111] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:18.980558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:18.981245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:18.981489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:19.007748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:19.009961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:19.085331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 554/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:19.109563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b22f-8cda-429c-8ff3-5111c093fae5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:19.142084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:19.224200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b22f-8cda-429c-8ff3-5111c093fae5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:19.266985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:19.268032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:19.280159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:19.280159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] PATH_INFO: `/v3/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:19.280159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] RBAC: Authorizing `identity:get_role(role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:19.280159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:19.280159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 559/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:19 2022] GET /identity/v3/roles/899a8afb374942c8872cc2929f6b4b1a => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:19.284904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:19.357092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:19.408855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da023053-e055-48ef-b26c-f79c66abeb94 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:19.445340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:19.446211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:19.446534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:19.446812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] PATH_INFO: `/v3/users/e1fc27a9766f430886ce67003e9a114c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:19.460668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] RBAC: Authorizing `identity:get_user(user_id=e1fc27a9766f430886ce67003e9a114c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:19.462122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:19.463797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 555/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:19 2022] GET /identity/v3/users/e1fc27a9766f430886ce67003e9a114c => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:19.471297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:19.566135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:19.577361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4cbfe-d456-47b2-aaf3-d040dc6c20c9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:19.614757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:19.615391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:19.615668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:19.615942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:19.687161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:19.688603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:19.792054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:19.807630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 560/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:19.816995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:19.922062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a14b-a9fb-410d-a62c-ae9f9864dffb None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:19.942528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:19.957673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:19.958551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:19.958833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:19.959103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:19.960327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:19.961748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652e588-eeeb-4d87-bb04-08f53f69825a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:19.963460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 556/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:19 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:19.969760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:20.073269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f82c75-db1a-4145-9495-1ef664e38225 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:20.108127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:20.108864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:20.109143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:20.109420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:20.110253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=3d5dec4686d84b9183d6154c5c515418, user.id=e1fc27a9766f430886ce67003e9a114c) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:20.110592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:20.112007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e291190-d428-41ed-8dd1-b8f12c45812e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:20.117644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 561/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:19 2022] GET /identity/v3/role_assignments?user.id=e1fc27a9766f430886ce67003e9a114c&scope.project.id=3d5dec4686d84b9183d6154c5c515418&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:22.074657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fb1fa-e2f5-4b65-ab78-fdb8e566c559 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.075236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fb1fa-e2f5-4b65-ab78-fdb8e566c559 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.075236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fb1fa-e2f5-4b65-ab78-fdb8e566c559 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.076650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 557/1118] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:22.083042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.083779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.083915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.106076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:22.107298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:22.163690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:22.183637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 562/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:22.206572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fb1fa-e2f5-4b65-ab78-fdb8e566c559 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:22.317050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fb1fa-e2f5-4b65-ab78-fdb8e566c559 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:22.352478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:22.353266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.353548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.353819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] PATH_INFO: `/v3/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.355438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] RBAC: Authorizing `identity:get_role(role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:22.356808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:22.358235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 558/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:22 2022] GET /identity/v3/roles/899a8afb374942c8872cc2929f6b4b1a => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:22.364486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:22.474821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ccbd-6719-4bde-91b9-e1fcdef0e9f3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:22.512776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:22.513647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.514068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.514212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] PATH_INFO: `/v3/users/e1fc27a9766f430886ce67003e9a114c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.515887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fc27a9766f430886ce67003e9a114c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:22.517325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:22.518962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 563/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:22 2022] GET /identity/v3/users/e1fc27a9766f430886ce67003e9a114c => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:22.528821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:22.595037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:22.642805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728228c-6387-4d3d-89e2-147fb482bbf5 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:22.679409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:22.680267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.680545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.680819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.682051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:22.683553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484457c1-7bc6-4627-825e-65d79cadfd3d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:22.685274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 559/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:22 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:22.692393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:22.760796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:22.808636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf7fb-886e-4818-9cac-3e57955a0c06 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:22.845380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:22.846261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:22.846568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:22.846841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418/users/e1fc27a9766f430886ce67003e9a114c/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:22.849396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d5dec4686d84b9183d6154c5c515418, user_id=e1fc27a9766f430886ce67003e9a114c, role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:22.850867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61190c5-f666-4612-82b0-8dd5e30c04d8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:22.867171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 564/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:03:22 2022] PUT /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418/users/e1fc27a9766f430886ce67003e9a114c/roles/899a8afb374942c8872cc2929f6b4b1a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:03:24.860964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719744-9560-4cab-89c3-0f77f495b8a5 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:24.861836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719744-9560-4cab-89c3-0f77f495b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:24.861836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719744-9560-4cab-89c3-0f77f495b8a5 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:24.864112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 560/1124] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:24.872708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:24.873180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:24.873430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:24.896333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:24.897450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:24.966927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 565/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:24.990287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55719744-9560-4cab-89c3-0f77f495b8a5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:25.105130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55719744-9560-4cab-89c3-0f77f495b8a5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:25.132395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:25.150756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:25.151962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:25.152388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:25.152837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] PATH_INFO: `/v3/roles/899a8afb374942c8872cc2929f6b4b1a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:25.155306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] RBAC: Authorizing `identity:get_role(role_id=899a8afb374942c8872cc2929f6b4b1a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:25.156950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:25.158337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 561/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:24 2022] GET /identity/v3/roles/899a8afb374942c8872cc2929f6b4b1a => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:25.164297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:25.215388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:25.282168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc08e5-913b-4fa0-ae83-755495ced326 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:25.318307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:25.319219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:25.319496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:25.319773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] PATH_INFO: `/v3/users/e1fc27a9766f430886ce67003e9a114c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:25.320977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fc27a9766f430886ce67003e9a114c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:25.322416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:25.324100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 566/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:25 2022] GET /identity/v3/users/e1fc27a9766f430886ce67003e9a114c => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:25.330272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:25.448610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1c43c-4f89-4612-a400-dae262b9cc14 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:25.495016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:25.495845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:25.496361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:25.496800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:25.572664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:25.585093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:25.586705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:25.698605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 562/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:25.706153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:25.848416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec809340-afb8-4af3-9e02-836afd23d8e3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:25.864564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:25.888923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:25.889983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:25.890161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:25.890344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] PATH_INFO: `/v3/projects/3d5dec4686d84b9183d6154c5c515418` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:25.891712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] RBAC: Authorizing `identity:get_project(project_id=3d5dec4686d84b9183d6154c5c515418)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:25.893161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe056-7672-4c6b-9023-b0bfeb23ac16 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:25.894929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 567/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:25 2022] GET /identity/v3/projects/3d5dec4686d84b9183d6154c5c515418 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:25.901899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:26.028782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:26.040044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4984-d696-4a93-b9cf-a4f506342b99 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:26.077404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:26.078094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:26.078351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:26.078679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:26.079497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] RBAC: Adding query filter params (role.id=899a8afb374942c8872cc2929f6b4b1a, scope.project.id=3d5dec4686d84b9183d6154c5c515418, user.id=e1fc27a9766f430886ce67003e9a114c) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:26.079808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:26.081230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8698-e26b-48de-a493-5737cc9a5609 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:26.087827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 563/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:25 2022] GET /identity/v3/role_assignments?user.id=e1fc27a9766f430886ce67003e9a114c&scope.project.id=3d5dec4686d84b9183d6154c5c515418&role.id=899a8afb374942c8872cc2929f6b4b1a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:28.127306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2883-bdb5-4ce8-b7f6-0d5676b00c7a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:28.128211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2883-bdb5-4ce8-b7f6-0d5676b00c7a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:28.128211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2883-bdb5-4ce8-b7f6-0d5676b00c7a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:28.130344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 568/1131] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:28.136022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:28.136457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:28.136556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:28.155006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:28.156235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:28.228001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 564/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:03:28.261231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed2883-bdb5-4ce8-b7f6-0d5676b00c7a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:28.372652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed2883-bdb5-4ce8-b7f6-0d5676b00c7a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:28.420072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:28.420796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:28.421064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:28.421321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:28.422602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:28.424035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06955947-dffd-4f4f-9f30-ae7dcabe4e49 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:28.425638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 569/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:28.434143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:28.520487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:28.536957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d961-bd95-4282-be9d-1a3cad01a091 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:28.571907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:28.572557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:28.572836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:28.573110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:28.573817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:28.575255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a98d70-19de-4639-9383-bac002b997b4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:28.602329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 565/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:03:28 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:30.617765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc623bf-7f93-42d7-ac3e-20fd1d68b9c0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:30.618669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc623bf-7f93-42d7-ac3e-20fd1d68b9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:30.618669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc623bf-7f93-42d7-ac3e-20fd1d68b9c0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:30.620874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 570/1135] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:30.626296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:30.626788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:30.626888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:30.646574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:30.647767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:30.719734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 566/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:30.744809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc623bf-7f93-42d7-ac3e-20fd1d68b9c0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:30.807055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:30.866222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc623bf-7f93-42d7-ac3e-20fd1d68b9c0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:30.903284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:30.904011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:30.904304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:30.904580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:30.906312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:30.907784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b56c6-d17d-4638-a70f-34e2c7b2eebd None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:30.909394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 571/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:03:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:30.916736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:30.984463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:31.024799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05b84e-4eb6-4b52-b84e-7d4abcd6e1e2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:31.063802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:31.064462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:31.064749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:31.065020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:31.065725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:31.067182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991886f-4ea6-43d2-9502-05b461435fe6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:31.089234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 567/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:03:30 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:33.127368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebc4c7-0d98-40fe-9473-b96afd8d6445 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.128214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebc4c7-0d98-40fe-9473-b96afd8d6445 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.128214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebc4c7-0d98-40fe-9473-b96afd8d6445 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:33.130417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 572/1139] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:33.137070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.137737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.137930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:33.158267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:33.159501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:33.231010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 568/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:33.269440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebc4c7-0d98-40fe-9473-b96afd8d6445 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:33.400111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebc4c7-0d98-40fe-9473-b96afd8d6445 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:33.417672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:33.439640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:33.440418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.440705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.440975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:33.451085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:33.458795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:33.460035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:33.461602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 573/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:33.469134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:33.527330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:33.586017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8466f74-bcfd-4b41-911c-93ebc0e27ecf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:33.620123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:33.620783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.621063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.621340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:33.622081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:33.622415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:33.623777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:33.632506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 569/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:33.638875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:33.747051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd130-51af-4499-8d04-f14821685d0c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:33.779390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:33.780245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.780554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.780826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] PATH_INFO: `/v3/users/6b71f71e901443c4946ffd11873e3ed2` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:33.794084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] RBAC: Authorizing `identity:get_user(user_id=6b71f71e901443c4946ffd11873e3ed2)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:33.795551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:33.797217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 574/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:33 2022] GET /identity/v3/users/6b71f71e901443c4946ffd11873e3ed2 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:33.803984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:33.914528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48dd1b0-3028-4a2d-abb0-9eed40699d0e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:33.931206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:33.957397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:33.958044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:33.958325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:33.958638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:34.027853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:34.029245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:34.131085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 570/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:34.138204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:34.175580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:34.242592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b4a2-e6a5-4836-8754-659c731ecd30 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:34.277605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:34.278548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:34.278769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:34.279044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] PATH_INFO: `/v3/projects/2e8f8a891c26461cac4853d16fe5b589` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:34.280350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] RBAC: Authorizing `identity:get_project(project_id=2e8f8a891c26461cac4853d16fe5b589)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:34.281781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537afaa-8e34-4c90-86b6-14e8d86ab99d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:34.283493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 575/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:34 2022] GET /identity/v3/projects/2e8f8a891c26461cac4853d16fe5b589 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:34.289655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:34.308501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:34.403658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912b43-fa6a-4fe8-a17a-bd18526bd5dd None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:34.440036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:34.440682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:34.440967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:34.441246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:34.442118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=2e8f8a891c26461cac4853d16fe5b589, user.id=6b71f71e901443c4946ffd11873e3ed2) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:34.442422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:34.443870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9601c-b9cc-407b-abb8-b04fa13a4c12 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:34.449631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 571/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:34 2022] GET /identity/v3/role_assignments?user.id=6b71f71e901443c4946ffd11873e3ed2&scope.project.id=2e8f8a891c26461cac4853d16fe5b589&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:36.456544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f371-d2ec-4094-ac62-2515cb7ecb5a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:36.457390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f371-d2ec-4094-ac62-2515cb7ecb5a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:36.457390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f371-d2ec-4094-ac62-2515cb7ecb5a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:36.459624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 576/1147] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:36.468618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:36.469056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:36.469223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:36.497298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:36.499153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:36.571882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 572/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:36.602923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f371-d2ec-4094-ac62-2515cb7ecb5a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:36.722697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f371-d2ec-4094-ac62-2515cb7ecb5a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:36.724163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:36.759801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:36.760586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:36.760863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:36.761135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:36.772405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:36.779951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:36.781114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:36.782207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 577/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:36.788202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:36.802888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:36.892734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aed3f0-4707-4991-ac3d-43302e32dcb2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:36.928245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:36.928898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:36.929177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:36.929453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:36.930201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:36.930543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:36.931910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:36.940184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 573/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:36.946497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:37.064446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bdbf4-b55f-45b6-853c-8c25a10a9de1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:37.100983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:37.101863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:37.102146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:37.102415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] PATH_INFO: `/v3/users/6b71f71e901443c4946ffd11873e3ed2` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:37.103708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] RBAC: Authorizing `identity:get_user(user_id=6b71f71e901443c4946ffd11873e3ed2)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:37.105155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:37.106831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 578/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:36 2022] GET /identity/v3/users/6b71f71e901443c4946ffd11873e3ed2 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:37.116263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:37.206106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:37.238109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c130-d51c-42a2-b244-c9b705bb0ce1 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:37.289653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:37.290554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:37.290842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:37.291118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] PATH_INFO: `/v3/projects/2e8f8a891c26461cac4853d16fe5b589` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:37.292527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] RBAC: Authorizing `identity:get_project(project_id=2e8f8a891c26461cac4853d16fe5b589)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:37.293972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761870e-927f-459b-9e33-84951c5c738d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:37.295757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 574/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:37 2022] GET /identity/v3/projects/2e8f8a891c26461cac4853d16fe5b589 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:37.302624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:37.341012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:37.421019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca8f60-022c-4605-9de4-5f8a042024dc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:37.470550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:37.471436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:37.471712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:37.471984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] PATH_INFO: `/v3/projects/2e8f8a891c26461cac4853d16fe5b589/users/6b71f71e901443c4946ffd11873e3ed2/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:37.474073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e8f8a891c26461cac4853d16fe5b589, user_id=6b71f71e901443c4946ffd11873e3ed2, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:37.475531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d89f96-c5c5-4eb2-be7c-fe2d893dbc89 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:37.491592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 579/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:03:37 2022] PUT /identity/v3/projects/2e8f8a891c26461cac4853d16fe5b589/users/6b71f71e901443c4946ffd11873e3ed2/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:03:39.452525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a825596-685d-448f-89e4-a8dafa3ac5bd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:39.453380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a825596-685d-448f-89e4-a8dafa3ac5bd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:39.453380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a825596-685d-448f-89e4-a8dafa3ac5bd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:39.455702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 575/1154] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:39.461818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:39.462569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:39.462784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:39.482757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:39.483989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:39.554809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 580/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:39.578878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a825596-685d-448f-89e4-a8dafa3ac5bd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:39.706328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a825596-685d-448f-89e4-a8dafa3ac5bd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:39.741590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:39.742415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:39.742696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:39.742974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:39.751472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:39.756711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:39.768240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:39.769367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:39.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 576/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:39.777366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:39.801914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:39.885613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d164dc-e9a0-43ff-8397-53dd88ec1a51 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:39.920950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:39.921603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:39.921883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:39.922161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:39.922954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:39.923265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:39.924623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:39.933923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 581/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:39.942978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:40.047577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9ae7-c0b0-45c7-98a9-3e7172cb58ed None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:40.083049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:40.083904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:40.084181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:40.084456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] PATH_INFO: `/v3/users/6b71f71e901443c4946ffd11873e3ed2` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:40.085902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] RBAC: Authorizing `identity:get_user(user_id=6b71f71e901443c4946ffd11873e3ed2)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:40.087367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:40.088985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 577/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:39 2022] GET /identity/v3/users/6b71f71e901443c4946ffd11873e3ed2 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:40.095406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:40.180284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:40.199759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8626-5e10-4263-8969-3729c2bfd08f None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:40.234577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:40.235215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:40.235493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:40.235766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:40.303611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:40.304972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:40.388824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:40.409113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 582/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:40.415296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:40.482941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:40.519845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2b5f-9a52-4b6d-8d77-07d19cebd5ef None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:40.555870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:40.556638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:40.556916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:40.557191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] PATH_INFO: `/v3/projects/2e8f8a891c26461cac4853d16fe5b589` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:40.558388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] RBAC: Authorizing `identity:get_project(project_id=2e8f8a891c26461cac4853d16fe5b589)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:40.559811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d58c9-f1fa-42cb-850b-eb2da7af0365 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:40.561532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 578/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:40 2022] GET /identity/v3/projects/2e8f8a891c26461cac4853d16fe5b589 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:40.568245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:40.684685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b81be-e018-4ace-a95b-e03fb352a550 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:40.720263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:40.720911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:40.721189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:40.721463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:40.722284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=2e8f8a891c26461cac4853d16fe5b589, user.id=6b71f71e901443c4946ffd11873e3ed2) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:40.722622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:40.724049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b7f66-8c6a-46b1-af4a-1dc960e1977b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:40.729940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 583/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:40 2022] GET /identity/v3/role_assignments?user.id=6b71f71e901443c4946ffd11873e3ed2&scope.project.id=2e8f8a891c26461cac4853d16fe5b589&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:42.766726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa175f0-9786-4078-b73b-8be43f4550d1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:42.767303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa175f0-9786-4078-b73b-8be43f4550d1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:42.767303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa175f0-9786-4078-b73b-8be43f4550d1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:42.768663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 579/1162] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:42.774992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:42.775733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:42.775867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:42.801253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:42.802460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:42.846714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:42.872581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 584/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:42.902347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa175f0-9786-4078-b73b-8be43f4550d1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:43.012816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa175f0-9786-4078-b73b-8be43f4550d1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:43.067377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:43.068024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:43.068303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:43.068579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:43.072285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:43.151840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:43.153350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:43.253582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 580/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:43.263365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:43.386834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272910a-2b3c-4706-8e2c-475d48caafb6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:43.431030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:43.431771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:43.432059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:43.432332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:43.439667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:43.441112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9c5b1374-450a-44e2-8543-7e549bd4edc9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:43.455625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:43.456789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:43.458708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 585/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:03:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:43.466316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:43.506431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:43.573017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ca300-34a7-4863-a3e5-c36987e8450a None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:43.608504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:43.608857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:43.609138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:43.609408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:43.610116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:43.610449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:43.611791 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684ba95-3cca-428b-95bc-9ff2b02ae11c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:43.619188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 581/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Mon Jul 4 04:03:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:03:45.626779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30de06-a3bd-494e-a052-f563ddd929d6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:45.627329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30de06-a3bd-494e-a052-f563ddd929d6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:45.627415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30de06-a3bd-494e-a052-f563ddd929d6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:45.629450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 586/1167] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:45.635161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ddbe9-b0fc-4995-85c2-8cd5d85608ab None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:45.635586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ddbe9-b0fc-4995-85c2-8cd5d85608ab None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:45.635703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ddbe9-b0fc-4995-85c2-8cd5d85608ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:45.654542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-632ddbe9-b0fc-4995-85c2-8cd5d85608ab None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:45.655895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-632ddbe9-b0fc-4995-85c2-8cd5d85608ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:45.723934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 582/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:45.747726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30de06-a3bd-494e-a052-f563ddd929d6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:45.793963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:45.864127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30de06-a3bd-494e-a052-f563ddd929d6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:45.899690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:45.900271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:45.900547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:45.900804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:45.901550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:45.902954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e0361-fabe-42d4-ae54-b970589830a5 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:45.920137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 587/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Mon Jul 4 04:03:45 2022] POST /identity/v3/domains => generated 267 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:47.950241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968db3-90bf-4351-a4dc-0663960e233c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:47.950801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968db3-90bf-4351-a4dc-0663960e233c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:47.950801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968db3-90bf-4351-a4dc-0663960e233c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:47.952138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 583/1170] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:47.960831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:47.961523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:47.961663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:47.986845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:47.988017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:48.056605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 588/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:48.080541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2968db3-90bf-4351-a4dc-0663960e233c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:48.122739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:48.199355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2968db3-90bf-4351-a4dc-0663960e233c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:48.234915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:48.235699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:48.235984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:48.236255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] PATH_INFO: `/v3/domains/28cff5db8db64d17aa2dcf5032a45bb3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:48.247460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=28cff5db8db64d17aa2dcf5032a45bb3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:48.248962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf06364-0e35-4bdb-ad46-36e3ae4aa4bd None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:48.250611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 584/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Jul 4 04:03:48 2022] GET /identity/v3/domains/28cff5db8db64d17aa2dcf5032a45bb3 => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:48.256519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:48.284219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:48.360860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa778-5601-4de9-a28c-19e950e9c558 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:48.397198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:48.397850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:48.398141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:48.398451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:48.399154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:48.400565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2361b04-7c56-4e08-9a1c-4589861fe486 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:48.427900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 589/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:03:48 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:50.454746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232690c6-1f02-4a32-b4ac-13d5762efafd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:50.455426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232690c6-1f02-4a32-b4ac-13d5762efafd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:50.455426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232690c6-1f02-4a32-b4ac-13d5762efafd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:50.456657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 585/1174] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:50.462961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:50.463701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:50.463845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:50.488412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:50.489577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:50.560732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 590/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:50.585368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232690c6-1f02-4a32-b4ac-13d5762efafd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:50.688406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:50.706549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232690c6-1f02-4a32-b4ac-13d5762efafd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:50.743627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:50.744493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:50.744778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:50.745056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] PATH_INFO: `/v3/domains/28cff5db8db64d17aa2dcf5032a45bb3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:50.746351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] RBAC: Authorizing `identity:get_domain(domain_id=28cff5db8db64d17aa2dcf5032a45bb3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:50.747809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784542e-c6aa-4307-8c94-36e2ccd1582e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:50.749711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 586/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Mon Jul 4 04:03:50 2022] GET /identity/v3/domains/28cff5db8db64d17aa2dcf5032a45bb3 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:50.759412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:50.775930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:50.872594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0939c-2e23-4619-97a0-d7c9c82ff146 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:50.912262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:50.913216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:50.913580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:50.913952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:50.915089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:50.917331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf5033-53e7-4d13-91d5-249471239405 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:50.946503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 591/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:03:50 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:03:52.986956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781dee3-acba-427c-b1ea-97f96d6acd91 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:52.987795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781dee3-acba-427c-b1ea-97f96d6acd91 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:52.987795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781dee3-acba-427c-b1ea-97f96d6acd91 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:52.989930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 587/1178] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:52.995303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:52.995745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:52.995844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:53.014605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:53.015773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:53.076180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:53.086098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 592/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:53.109231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781dee3-acba-427c-b1ea-97f96d6acd91 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:53.227519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781dee3-acba-427c-b1ea-97f96d6acd91 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:53.263204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:53.264054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:53.264344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:53.264614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:53.276364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:53.283965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:53.285124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:53.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 588/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:53.294519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:53.394477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c378f-5af2-4300-8268-893f5ef74396 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:53.429579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:53.430236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:53.430545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:53.430817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:53.431564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:53.431871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:53.433224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:53.441606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 593/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:53.447530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:53.469123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:53.555479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28339-cde4-4175-8118-9365acdf0787 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:53.590654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:53.591561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:53.591868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:53.592198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] PATH_INFO: `/v3/users/798ae1432e494452b60964ed86946ebd` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:53.605847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] RBAC: Authorizing `identity:get_user(user_id=798ae1432e494452b60964ed86946ebd)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:53.607312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:53.608986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 589/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:53 2022] GET /identity/v3/users/798ae1432e494452b60964ed86946ebd => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:53.615499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:53.678824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:53.726597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc85137-6390-4f47-aca2-0919bacc9f01 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:53.763452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:53.764138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:53.764447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:53.764823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:53.834074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:53.835484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:53.891291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:53.937683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 594/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:03:53.943748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:54.022983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:54.049303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bbdd-3359-4cbe-b9a6-54a18090aa46 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:54.084548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:54.085338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:54.085619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:54.085899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] PATH_INFO: `/v3/projects/c3e43e999fb94aeda19b09ee456b79e8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:54.087122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] RBAC: Authorizing `identity:get_project(project_id=c3e43e999fb94aeda19b09ee456b79e8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:54.088557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afe4ca-8210-4596-b0db-4783a238bf70 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:54.090258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 590/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:53 2022] GET /identity/v3/projects/c3e43e999fb94aeda19b09ee456b79e8 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:54.098401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:54.202506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d1d90-6a9f-473d-9d34-7dec002cf0d1 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:54.237457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:54.238099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:54.238413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:54.238685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:54.239511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=c3e43e999fb94aeda19b09ee456b79e8, user.id=798ae1432e494452b60964ed86946ebd) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:54.239823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:54.241238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dff5e1-fb2d-40eb-a1d7-18475af78820 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:54.246821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 595/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:03:54 2022] GET /identity/v3/role_assignments?user.id=798ae1432e494452b60964ed86946ebd&scope.project.id=c3e43e999fb94aeda19b09ee456b79e8&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:56.220401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd90c-91da-4b7b-9b32-78f6bf072180 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:56.221013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd90c-91da-4b7b-9b32-78f6bf072180 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:56.221013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd90c-91da-4b7b-9b32-78f6bf072180 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:56.222343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 591/1186] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:56.228751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:56.229435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:56.229623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:56.249618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:56.250833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:56.267047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:56.330300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 596/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:56.364729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecd90c-91da-4b7b-9b32-78f6bf072180 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:56.495875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecd90c-91da-4b7b-9b32-78f6bf072180 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:56.530257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:56.531062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:56.531339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:56.531609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:56.542055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:56.549688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:56.550803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:03:56.551765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 592/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:56.559455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:56.670303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a56bc7-67dc-44ab-b23f-b83c72fc5a33 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:56.707431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:56.708090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:56.708373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:56.708653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:56.709415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:56.709720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:56.711136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:56.719404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 597/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:56.725475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:56.730003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:56.830925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4f4b-615a-43f1-a800-02b50666c077 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:56.883519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:56.884411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:56.884694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:56.884969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] PATH_INFO: `/v3/users/798ae1432e494452b60964ed86946ebd` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:56.886412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] RBAC: Authorizing `identity:get_user(user_id=798ae1432e494452b60964ed86946ebd)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:56.887903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:56.889616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 593/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:56 2022] GET /identity/v3/users/798ae1432e494452b60964ed86946ebd => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:56.895889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:56.912012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:57.000393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0717bb-8283-42ae-9964-490d6a770bfc None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:57.036397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:57.037243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:57.037520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:57.037796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] PATH_INFO: `/v3/projects/c3e43e999fb94aeda19b09ee456b79e8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:57.039099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] RBAC: Authorizing `identity:get_project(project_id=c3e43e999fb94aeda19b09ee456b79e8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:57.040622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf9b4a-2871-40cd-84db-bce4b2c2f073 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:57.042300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 598/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:03:56 2022] GET /identity/v3/projects/c3e43e999fb94aeda19b09ee456b79e8 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:57.049463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:57.143685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:57.187269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84caddcb-02ba-4f55-8dd1-a3127c0acda6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:57.235692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:57.236587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:57.236874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:57.237147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] PATH_INFO: `/v3/projects/c3e43e999fb94aeda19b09ee456b79e8/users/798ae1432e494452b60964ed86946ebd/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:57.239392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] RBAC: Authorizing `identity:create_grant(project_id=c3e43e999fb94aeda19b09ee456b79e8, user_id=798ae1432e494452b60964ed86946ebd, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:57.240834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362f07e-5686-4cb8-a5f2-0f9800c8665d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:57.255989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 594/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:03:57 2022] PUT /identity/v3/projects/c3e43e999fb94aeda19b09ee456b79e8/users/798ae1432e494452b60964ed86946ebd/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:03:59.303027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d826b-11c7-4102-bc65-e1a39c8336ea None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:59.303617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d826b-11c7-4102-bc65-e1a39c8336ea None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:59.303617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d826b-11c7-4102-bc65-e1a39c8336ea None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:59.304969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 599/1193] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:03:59.311188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:59.311653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:59.312030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:59.334097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:03:59.335332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:03:59.405271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 595/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:03:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:03:59.433681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d826b-11c7-4102-bc65-e1a39c8336ea None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:59.525135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:59.548687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d826b-11c7-4102-bc65-e1a39c8336ea None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:59.585232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:59.585953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:59.586239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:59.586508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:59.597652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:03:59.605338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:03:59.606436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:03:59.607516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 600/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:03:59.614122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:59.679722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:03:59.722741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191dc46-fb71-439f-b986-a43f3a198889 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:59.759685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:59.760351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:59.760640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:59.760916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:59.761671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:03:59.761981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:59.763385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:59.771854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 596/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:03:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:59.778267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:03:59.894327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c554bf0-71a3-49b0-8c59-8cd1257a6409 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:03:59.932067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74983920-2626-485e-930c-db503552ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:03:59.932947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74983920-2626-485e-930c-db503552ab12 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:03:59.933386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74983920-2626-485e-930c-db503552ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:03:59.933536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74983920-2626-485e-930c-db503552ab12 None admin] PATH_INFO: `/v3/users/798ae1432e494452b60964ed86946ebd` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:03:59.934723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74983920-2626-485e-930c-db503552ab12 None admin] RBAC: Authorizing `identity:get_user(user_id=798ae1432e494452b60964ed86946ebd)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:03:59.936428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74983920-2626-485e-930c-db503552ab12 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:03:59.938040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 601/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Mon Jul 4 04:03:59 2022] GET /identity/v3/users/798ae1432e494452b60964ed86946ebd => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:03:59.946507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:00.053246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ce11-be64-4721-8bf7-6aaee7a7953d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:00.077250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:00.089014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:00.089664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:00.089947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:00.090221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:00.163946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:00.165325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:00.268356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 597/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:03:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:04:00.274691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74983920-2626-485e-930c-db503552ab12 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:00.291112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:00.386883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74983920-2626-485e-930c-db503552ab12 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:00.421762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:00.422589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:00.422873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:00.423146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] PATH_INFO: `/v3/projects/c3e43e999fb94aeda19b09ee456b79e8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:00.424379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] RBAC: Authorizing `identity:get_project(project_id=c3e43e999fb94aeda19b09ee456b79e8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:00.425816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22061449-7b59-4179-9588-63c5c9c4fb39 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:00.427548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 602/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Mon Jul 4 04:04:00 2022] GET /identity/v3/projects/c3e43e999fb94aeda19b09ee456b79e8 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:04:00.437203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:00.467379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:00.552180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5fcc2-fc83-44d0-8fbb-03c483248c3e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:00.588559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:00.589210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:00.589492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:00.589767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:00.590718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=c3e43e999fb94aeda19b09ee456b79e8, user.id=798ae1432e494452b60964ed86946ebd) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:04:00.591069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:00.592466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5ef24-5fe2-4a46-b1a2-ce790e8b17e0 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:00.598419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 598/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:04:00 2022] GET /identity/v3/role_assignments?user.id=798ae1432e494452b60964ed86946ebd&scope.project.id=c3e43e999fb94aeda19b09ee456b79e8&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:04:10.416938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=62 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:10.595825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=56 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:55.956491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33898b-8dbb-4fbf-871f-1142f039c6f3 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:55.956491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33898b-8dbb-4fbf-871f-1142f039c6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:55.956491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33898b-8dbb-4fbf-871f-1142f039c6f3 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:55.956491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 603/1201] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:04:55.962623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:55.963115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:55.963205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:55.981451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:04:55.982735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:04:56.070534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 599/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:04:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:04:56.093934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33898b-8dbb-4fbf-871f-1142f039c6f3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:56.228182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33898b-8dbb-4fbf-871f-1142f039c6f3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:56.273613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:56.274616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:56.274931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:56.275213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:56.288495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:56.290059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:04:56.310210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:04:56.311456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:04:56.312448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:04:56.312448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:04:56.312448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:04:56.312448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:04:56.316454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 604/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:04:56.320055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:56.446956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f0992-768c-4554-9652-20b530264c49 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:56.488920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:56.496041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:56.496888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:56.497176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:56.497491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:56.498214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:04:56.498551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:56.499925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265e04c-e28b-4156-ae1f-a176376a0088 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:56.513528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 600/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:04:56.519220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:56.581490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:56.646004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1507e0-f0d9-424a-8d60-56c3e72fec22 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:56.687691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:56.688379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:56.688662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:56.688935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:56.689670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:56.691201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea22ed8-90af-4a1b-a5eb-654f7b5f9f24 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:56.717847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 605/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Mon Jul 4 04:04:56 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:04:58.946466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de69a-0ea1-4440-bd26-614c2115a5a7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:58.946987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de69a-0ea1-4440-bd26-614c2115a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:58.946987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de69a-0ea1-4440-bd26-614c2115a5a7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:58.948380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 601/1206] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:04:58.955523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:58.956364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:58.956600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:58.983736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:04:58.983736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:04:59.085170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 606/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:04:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:04:59.111344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de69a-0ea1-4440-bd26-614c2115a5a7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:59.233410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de69a-0ea1-4440-bd26-614c2115a5a7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:59.274770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:59.275610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:59.275914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:59.276163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:59.288700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:04:59.297237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:04:59.298310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:04:59.299270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 602/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:04:59.304807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:59.357110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:59.434362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ae81d-41e8-46e2-8b64-91cb2b87206a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:59.473062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:59.473766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:59.474050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:59.474298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:59.475112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:04:59.475416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:59.476766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:59.491180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 607/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:04:59.492553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:59.556648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:59.624058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1921a9-4f01-47f4-ae7f-34dcc5d835b7 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:59.673286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:59.674128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:59.674452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:59.674737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:59.686630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:59.688096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:04:59.708060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:04:59.709364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:04:59.710661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:04:59.710661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:04:59.710661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:04:59.710661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:04:59.710661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 603/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:04:59.715385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:04:59.828664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:04:59.832509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b095-61eb-4dcd-b950-a6c3c93a9e18 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:04:59.876249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:04:59.876973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:04:59.877261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:04:59.877571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:04:59.878480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:04:59.878834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:04:59.880390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:04:59.895179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 608/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:04:59.900969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:00.017216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3699-1396-4190-b116-c5c182463c0f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:00.019064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:00.061970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:00.062852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:00.063127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:00.063396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:00.072424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:00.073965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:00.090630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:00.091704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:00.093308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 604/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:04:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:00.097919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:00.215024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da8903-81d9-4b66-a5cf-4b424ed5f4f2 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:00.251708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:00.252415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:00.252674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:00.252956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:00.253930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:00.254188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:00.255947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:00.278267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 609/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:00.284062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:00.413325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f711-b595-446a-87e7-67bdefadb2a3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:00.456331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:00.457144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:00.457567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:00.457892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:00.498564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:00.548958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:00.550939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:00.684539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 605/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:05:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:05:00.685831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:00.708666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:00.812608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414cce9-19eb-4c4c-aa48-e1b6b6455141 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:00.856402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:00.857749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:00.858203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:00.858667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:00.872254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:00.873798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:00.895083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:00.896989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:00.898218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:00.898218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:00.898218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:00.898218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:00.898218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 610/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:00.902054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:00.969669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:01.019967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e1a1c-ad09-48ce-84e7-a0f9b59043c6 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:01.070799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:01.071902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:01.072376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:01.072848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:01.074238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:01.074799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:01.077198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:01.097425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 606/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:01.102937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:01.184405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:01.218338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335702d-1b13-447d-a6f1-0605890fdb87 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:01.258305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:01.259158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:01.259413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:01.259695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:01.268042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:01.269594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:01.285300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:01.286494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:01.287713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:01.291567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 611/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:05:01.292523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:01.440590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ac258-a477-4cc4-87a2-2639ef7bb98e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:01.463967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:01.477370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:01.478189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:01.478524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:01.478885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:01.479691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:01.480142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:01.481645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cf06-7c19-4a13-be76-70a93b22a93c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:01.494282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 607/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:01.500130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:01.618433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b528bc-3d71-49f4-b839-f4edfc2c30a9 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:01.654221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:01.660964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:01.662211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:01.662929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:01.663555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:01.664591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01abfdc5d30744c89a476de93f537305) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:01.665182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:01.666936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d56b-d28a-4b5c-8cf5-c866e923ea7c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:01.673941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 612/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:05:01 2022] GET /identity/v3/role_assignments?user.id=01abfdc5d30744c89a476de93f537305&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:03.730032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc2c0-2cf2-4d0a-ba5e-1f0ac5b5c903 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:03.732027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc2c0-2cf2-4d0a-ba5e-1f0ac5b5c903 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:03.732027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc2c0-2cf2-4d0a-ba5e-1f0ac5b5c903 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:03.732574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 608/1220] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:03.737996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:03.738605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:03.738605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:03.755948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:03.757459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:03.831114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 613/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:03.853358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc2c0-2cf2-4d0a-ba5e-1f0ac5b5c903 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:03.965891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc2c0-2cf2-4d0a-ba5e-1f0ac5b5c903 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.001524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.002323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.002633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.002907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.014147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:05:04.023260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:04.024536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:04.025578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 609/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:04.032724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:04.147931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed947d7-7eee-416f-93da-f9cb5a7f5861 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.174773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:04.186134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.186838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.187122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.187395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.188145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] RBAC: Adding query filter params (name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:04.188448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:04.189804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:04.197715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 614/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:04.203998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:04.252824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:04.314514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d99220-a06d-46f6-875c-2b417765ce30 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.355200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.356196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.356511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.356862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.369238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:04.370908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:04.388763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:04.389975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:04.390943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:04.390943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:04.390943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:04.390943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:04.390943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 610/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:04.396043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:04.512766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece6d95-5d7a-4d7f-a549-eaa2b400a64c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.547813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.548690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.549128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.549452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.550434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:04.550745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:04.552138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:04.565053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 615/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:04.571046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:04.679318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:04.688455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44077-03c9-4ca6-a259-4d9212e803bc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.725054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.725877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.726163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.726480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.734650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:04.736152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:04.750747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:04.751827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:04.752773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 611/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:04.758215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:04.808435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:04.863998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d1efc-ed5b-4fda-a686-d792ecb9392d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:04.902101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:04.902814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:04.903102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:04.903380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:04.904228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:04.904543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:04.905977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:04.924701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 616/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:04.930528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:05.039649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f899c-050e-4ba9-96d1-53a96f6b5662 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:05.076138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:05.076875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:05.077164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:05.077451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:05.089659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:05.091353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:05.109137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:05.110215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:05.111347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:05.111347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:05.111347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:05.111347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:05.111347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 612/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:05.117210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:05.225473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa1fc1-aec4-45d6-9660-6cf836f1b51d None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:05.228081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:05.262344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:05.263072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:05.263355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:05.263702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:05.264351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:05.264668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:05.266041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:05.277901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 617/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:05.284049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:05.294506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:05.396182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7ff9-de79-44f5-906d-4952a07b439b None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:05.444489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:05.445310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:05.445595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:05.445870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:05.453321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:05.454834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:05.469646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:05.470678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:05.471729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 613/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:05.477027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:05.584373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120443e2-4b01-4544-b33f-92c033999892 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:05.620779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:05.621574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:05.621866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:05.622164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:05.623004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:05.623369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:05.625268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb6d3-dd1d-45dd-8c0e-4c11ed762925 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:05.639610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 618/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:05.645506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:05.719126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:05.755963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d310da-a84f-48fb-ab7b-80a0dc813cfe None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:05.797156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:05.798070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:05.798353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:05.798662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01abfdc5d30744c89a476de93f537305/roles/c15d5032aba84fadbf10de688a735d87` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:05.814418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=01abfdc5d30744c89a476de93f537305, role_id=c15d5032aba84fadbf10de688a735d87)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:05.815994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837821c-34c8-422f-bb90-b3c254957700 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:05.829153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 614/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:05:05 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01abfdc5d30744c89a476de93f537305/roles/c15d5032aba84fadbf10de688a735d87 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:05:07.843605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7876f-fac7-4baf-9df7-36a3bfdc7519 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:07.844204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7876f-fac7-4baf-9df7-36a3bfdc7519 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:07.844204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7876f-fac7-4baf-9df7-36a3bfdc7519 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:07.845848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 619/1233] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:07.851147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:07.851569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:07.851749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:07.868523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:07.869708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:07.948034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 615/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:07.971656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7876f-fac7-4baf-9df7-36a3bfdc7519 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:07.998997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:08.086099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7876f-fac7-4baf-9df7-36a3bfdc7519 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:08.122722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:08.123657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:08.124059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:08.124342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:08.136315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:05:08.144100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:08.145083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:08.146247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 620/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Mon Jul 4 04:05:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:08.152216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:08.205859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:08.265532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec3290-554b-433c-9087-e13587e8d04f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:08.301050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:08.301725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:08.302013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:08.302286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:08.303072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:08.303379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:08.304748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:08.313334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 616/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Mon Jul 4 04:05:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:08.319305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:08.403735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:08.422246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779c94-9771-4e96-b751-ca836a94cebe None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:08.458963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:08.459692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:08.459978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:08.460258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:08.472108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:08.473663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:08.494083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:08.495471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:08.496626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:08.496626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:08.496626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:08.496626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:08.496626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 621/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:05:08.503035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:08.615686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894f61c-e6e0-4278-b9a9-5b94e25bda3d None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:08.626647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:08.653947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:08.654660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:08.654937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:08.655256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:08.655942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:08.656311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:08.657624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:08.669391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 617/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:08.675312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:08.784522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758312d-47af-485b-819f-9cb7ab325662 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:08.820511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:08.821328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:08.821621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:08.821892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:08.825296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:08.831081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:08.832607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:08.849894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:08.851524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:08.852690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 622/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:08.857278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:08.969951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3688b68-aef1-41ef-9eae-c413df222d95 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:09.009462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:09.010145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:09.010460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:09.010739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:09.011655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:09.011968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:09.013442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:09.032073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 618/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:09.038065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:09.145168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8e565-78cb-4cca-8fd2-ebaa083dd713 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:09.181348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:09.182024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:09.182317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:09.182656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:09.248605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:09.256753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:09.258715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:09.373228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 623/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:05:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:05:09.374653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:09.408256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:09.490800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845c92-c7c5-4b9d-b00d-c115a9bc8d96 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:09.527590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:09.528403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:09.528704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:09.528954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:09.540593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:09.542136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:09.562259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:09.563332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:09.564293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:09.564293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:09.564293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:09.564293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:09.564293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 619/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:09.571275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:09.678493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:09.685417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51538f10-7de0-4e35-827e-8c6f7912a00a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:09.725873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:09.726907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:09.727233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:09.727512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:09.728339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:09.728644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:09.730230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:09.742631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 624/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:09.748713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:09.831389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:09.857732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3840a-95ca-4da5-80ab-51a4ba09bc77 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:09.898117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:09.898857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:09.899190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:09.899635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:09.908362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:09.910045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:09.928518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:09.929632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:09.930748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:09.936207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 620/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:09.937527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:10.054191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d61dc7-fbf8-4e74-8bdc-caad13fc54c3 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:10.092517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:10.093202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:10.093484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:10.095376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:10.095376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:10.095376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:10.096321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09777f-607b-4042-8c49-340ac0b77a4b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:10.123894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 625/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:10.130074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:10.238906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b2e6e-c925-4853-9905-4d62b9c2cc42 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:10.274162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:10.274858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:10.275134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:10.275405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:10.276230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] RBAC: Adding query filter params (role.id=c15d5032aba84fadbf10de688a735d87, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01abfdc5d30744c89a476de93f537305) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:10.276600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:10.278059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0243d7-82d0-4339-8ce3-2cf5483ae97d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:10.284596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 621/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:05:10 2022] GET /identity/v3/role_assignments?user.id=01abfdc5d30744c89a476de93f537305&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=c15d5032aba84fadbf10de688a735d87 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:12.496545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a44b-75e4-42b6-9e7f-99c754a10a90 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:12.497211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a44b-75e4-42b6-9e7f-99c754a10a90 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:12.497211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a44b-75e4-42b6-9e7f-99c754a10a90 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:12.499976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 626/1247] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:12.504501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:12.504916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:12.505066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:12.511568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:12.525388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:12.526713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:12.603142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 622/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 04:05:12.628142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418a44b-75e4-42b6-9e7f-99c754a10a90 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:12.650520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:12.763896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418a44b-75e4-42b6-9e7f-99c754a10a90 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:12.807622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:12.808634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:12.808915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:12.809232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:12.820802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:05:12.828575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:12.829882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:12.831276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 627/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:12.835985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:12.945679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c8f0-f2d5-4e77-a52b-b4221b619fe8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:12.981630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:12.987627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:12.988412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:12.988727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:12.989006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:12.989811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:12.990182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:12.991720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:13.004478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 623/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:13.010768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:13.102461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:13.133433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178421-f459-4907-90cc-12a167b015d0 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:13.173858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:13.174710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:13.174995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:13.175272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:13.188221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:13.189732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:13.209001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:13.210015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:13.210982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:13.210982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:13.210982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:13.210982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:13.210982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 628/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:13.216536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:13.334422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61d9e-0ce4-4237-86d0-aa45617d88ba None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:13.372178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:13.372764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:13.373046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:13.373319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:13.374042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:13.374349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:13.375743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:13.388795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 624/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:13.394941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:13.527822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830d0b2-2f6f-4be3-8132-ada48a71385e None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:13.560781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:13.569753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:13.570862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:13.571143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:13.571413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:13.579598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:13.581071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:13.598770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:13.599882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:13.600833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 629/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:13.605938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:13.673604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:13.723717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834a8fa-3456-4b03-a7f1-c7d8c91c0ea3 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:13.763125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:13.763814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:13.764121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:13.764394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:13.765250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:13.765558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:13.767037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:13.786517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 625/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:13.792829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:13.912323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01977b-6538-4165-a62c-7151983ce63a None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:13.952147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:13.952803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:13.953098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:13.953380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:14.024638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:14.040361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:14.041865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:14.177377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 630/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:05:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:05:14.184331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:14.301325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef889-e272-4bba-94f5-bf655ec9068f None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:14.316396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:14.340460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:14.341328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:14.341618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:14.341895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:14.353031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:14.354649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:14.373231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:14.374180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:14.375599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:14.375599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:14.375599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:14.375599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:14.375599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 626/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:14.380694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:14.491677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:14.508348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9bbfe-dfbb-4b45-95aa-8991f75d1588 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:14.551876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:14.552606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:14.552808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:14.553091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:14.553802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:14.554105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:14.555575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:14.570554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 631/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:14.576515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:14.699615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606a52-6480-454c-975e-61da202ff602 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:14.736616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:14.737580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:14.737944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:14.738293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:14.746206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:14.747921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:14.763941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:14.765048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:14.771231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 627/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:14.771814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:14.890330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4a590-f0a9-4c68-99c3-52b577971b63 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:14.930588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:14.931259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:14.931547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:14.931822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:14.932713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:14.933034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:14.934583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c2bb3-0381-4402-a839-c98751ecda49 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:14.947046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 632/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:14.953534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:14.981002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:15.061306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc61-3684-4252-839a-28c169b60da9 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:15.099840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:15.100606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:15.100903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:15.101209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:15.102064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01abfdc5d30744c89a476de93f537305) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:15.102415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:15.103981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0491f-2732-450b-857f-4421c7745675 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:15.110934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 628/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:05:14 2022] GET /identity/v3/role_assignments?user.id=01abfdc5d30744c89a476de93f537305&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:17.112075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd71b-5b16-425e-bdf7-82f755d20e10 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.112973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd71b-5b16-425e-bdf7-82f755d20e10 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.112973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd71b-5b16-425e-bdf7-82f755d20e10 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.115188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 633/1261] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:17.121136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.121552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.121662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.139044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:17.140259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:17.211178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 629/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:17.241773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efd71b-5b16-425e-bdf7-82f755d20e10 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:17.253991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:17.353209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efd71b-5b16-425e-bdf7-82f755d20e10 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:17.389236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:17.390004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.390251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.390551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.402953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:05:17.410620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:17.412016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:17.413279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:17.417390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 634/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:17.418492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:17.439167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:17.528450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d0cbc-b10f-410e-bd35-c1a51a16fb25 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:17.570589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:17.571259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.571542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.571814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.572564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:17.572869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:17.574464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:17.582627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 630/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:17.589611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:17.673793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:17.711471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0cce5-8405-42dc-ace8-06537f406f17 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:17.746655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:17.747514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.747799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.748073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.759442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:17.761037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:17.778351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:17.779371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:17.780374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:17.780374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:17.780374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:17.780374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:17.780374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 635/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:17.786254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:17.884883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:17.911662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c4c47-c573-4386-b078-3b49e7221343 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:17.948617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:17.949218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:17.949500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:17.949771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:17.950528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:17.950837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:17.952212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:17.965514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 631/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:17.971801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.076214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ec63b-e335-4e1e-9936-2f0d8e905395 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:18.096464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:18.112451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:18.113277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:18.113565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:18.113871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:18.122312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:18.123775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:18.139492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:18.140673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:18.141877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:18.145842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 636/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:18.146789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.249730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8b53b-866b-46b2-a125-442432d89504 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:18.286995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:18.287697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:18.287976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:18.288250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:18.289086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:18.289366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:18.290826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:18.299764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:18.308854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 632/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:18.314818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.425550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc7c12-3b89-45f3-9987-194891c1b034 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:18.460419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:18.461207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:18.461485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:18.461758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:18.472799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:18.474224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:18.491728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:18.492735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:18.493733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:18.493733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:18.493733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:18.493733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:18.493733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 637/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 04:05:18.498923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.611240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5b3f8-b3cd-482a-bf67-f2b551860347 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:18.650168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:18.650862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:18.651272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:18.651804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:18.652164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:18.652575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:18.653943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:18.665250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 633/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:18.671317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.708952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:18.782902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf056f-2d8d-4583-a8db-0599be9d79b8 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:18.818914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:18.819705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:18.819993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:18.820271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:18.827650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:18.829094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:18.844044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:18.845134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:18.846194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 638/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:18.851322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:18.917052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:18.962116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5acd20-b5f5-4894-ba0d-56773ad9aaa2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:19.000631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:19.001273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:19.001561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:19.001837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:19.002655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:19.002970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:19.004423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06526dd-2ef1-4654-9c83-b148b0c62a9f None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:19.017402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 634/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:19.024094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:19.123860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:19.139736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd953bc-f070-4074-a1d8-bb617574964c None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:19.180210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:19.181014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:19.181291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:19.181561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] PATH_INFO: `/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01abfdc5d30744c89a476de93f537305/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:19.183739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] RBAC: Authorizing `identity:create_grant(project_id=bea1bc67780a4c32882810f1c0ffcb6a, user_id=01abfdc5d30744c89a476de93f537305, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:19.185168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f026c-8665-44aa-aa4e-a0aabfbd8017 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:19.201620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 639/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Mon Jul 4 04:05:19 2022] PUT /identity/v3/projects/bea1bc67780a4c32882810f1c0ffcb6a/users/01abfdc5d30744c89a476de93f537305/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:05:21.197271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec5f74-b29e-4584-8c21-56815384bebc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:21.197934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec5f74-b29e-4584-8c21-56815384bebc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:21.197934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec5f74-b29e-4584-8c21-56815384bebc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:21.199364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 635/1274] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:21.205817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:21.206577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:21.206577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:21.233002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:21.234191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:21.306979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 640/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:21.343507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec5f74-b29e-4584-8c21-56815384bebc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:21.458883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec5f74-b29e-4584-8c21-56815384bebc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:21.478904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:21.497822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:21.498650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:21.498936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:21.499212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:21.510016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 04:05:21.518149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:21.519258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:21.520228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 636/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Mon Jul 4 04:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:21.525474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:21.613399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:21.636918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d82e5-7ac0-4e17-beca-700df657c834 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:21.671423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:21.672091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:21.672378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:21.672653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:21.673408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:21.673718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:21.675106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:21.683248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 641/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Mon Jul 4 04:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:21.689110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:21.795629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f4298-5484-4378-aab7-6f17d361389e None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:21.832299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:21.833082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:21.833364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:21.833644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:21.845170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:21.846637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:21.864424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:21.865436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:21.866544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:21.866544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:21.866544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:21.866544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:21.870337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 637/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:21.872226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:21.986480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6bb74-28dc-4174-99e4-7eeeb35f557b None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:22.021748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:22.022413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:22.022699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:22.022978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:22.023699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:22.024011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:22.025372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:22.037140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 642/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:22.043052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:22.077543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:22.153685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21536ab-ffab-4b6c-95a4-a3003f44df28 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:22.191344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:22.192134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:22.192416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:22.192693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:22.200437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:22.201867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 04:05:22.216541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:22.217618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:22.218623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 638/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:22.225117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:22.231160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:22.339068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5006c-bd26-4b48-81d1-eae37efb7e94 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:22.374838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:22.375498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:22.375777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:22.376045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:22.376873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:22.377176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:22.378621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:22.397050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 643/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Mon Jul 4 04:05:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:22.406647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:22.506577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:22.521621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a637f-a91e-43f3-9a2a-f7a3317792ab None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:22.565379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:22.566031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:22.566316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:22.566623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:22.646040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:22.647544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:22.737894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:22.760700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 639/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Mon Jul 4 04:05:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 04:05:22.767590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:22.832923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:22.877753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65309e9-0016-4611-85ef-2b3b0f5fb873 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:22.913014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:22.913727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:22.914007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:22.914279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:22.924919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:22.926354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 04:05:22.944176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:22.945185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:22.946498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 04:05:22.946498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 04:05:22.946498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 04:05:22.946498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:22.946498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 644/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:22.953983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:23.070416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109bec6-e806-4ab5-92f2-9c41fe83c3cc None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:23.106313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:23.106996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:23.107277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:23.107550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:23.108274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:23.108582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:23.109950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:23.121509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 640/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Mon Jul 4 04:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:23.128047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:23.239257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc708f0-8287-4090-885a-8517d7acfa49 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:23.253724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:23.288922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:23.290083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:23.290445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:23.290846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:23.298615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:23.300046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 04:05:23.314359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:23.315485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: ERROR keystone.server.flask.application Jul 04 04:05:23.316606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 645/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Mon Jul 4 04:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:23.321657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:23.391280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:23.451088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64653-3346-4b3e-965b-8d3a230801c2 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:23.486753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:23.487412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:23.487698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:23.487978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:23.488775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:23.489088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:23.490627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020eb34-fabf-4f1c-be4a-a74be1e63cef None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:23.505273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 641/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Mon Jul 4 04:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:23.514210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:23.637956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332dce2-556c-42c7-b65a-db68cec85c35 None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:23.674512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:23.675157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:23.675437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:23.675714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:23.676549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] RBAC: Adding query filter params (role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40, scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a, user.id=01abfdc5d30744c89a476de93f537305) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:23.676857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:23.678292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263f949-0549-4810-a129-b1e12df24d55 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:23.684314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 646/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Mon Jul 4 04:05:23 2022] GET /identity/v3/role_assignments?user.id=01abfdc5d30744c89a476de93f537305&scope.project.id=bea1bc67780a4c32882810f1c0ffcb6a&role.id=ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:25.691022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ad02-4a03-4bdb-bc92-906a3cfe8b3e None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:25.691602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ad02-4a03-4bdb-bc92-906a3cfe8b3e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:25.691602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ad02-4a03-4bdb-bc92-906a3cfe8b3e None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:25.692944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 642/1288] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:25.698644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:25.699111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:25.699218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:25.722198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:25.723457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:25.758408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:25.804785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 647/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:25.827929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54ad02-4a03-4bdb-bc92-906a3cfe8b3e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:25.944875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54ad02-4a03-4bdb-bc92-906a3cfe8b3e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:25.961350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:25.988531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:25.989338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:25.989621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:25.989898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:25.990592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:25.991969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:26.004201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:26.005406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:26.006263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 643/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:05:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:26.012015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:26.115010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc9f9-a55e-4379-bbb3-800901a64ffd None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:26.152098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:26.153097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:26.153481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:26.153928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:26.155099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:26.155549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:26.157677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e768e-c127-40f5-a833-d6efd7e73cb8 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:26.166516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 648/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Jul 4 04:05:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:26.173126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:26.285720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941843f-bc69-44bf-8175-2f79c4f42970 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:26.321754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:26.322469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:26.322841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:26.323112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] PATH_INFO: `/v3/services` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:26.323837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:26.324141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:26.325503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99054d-555f-4ae5-8351-052b7bd73e7c None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:26.330144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 644/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Jul 4 04:05:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:28.302602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13d0bc-f200-4940-a81c-a6401c216dd4 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:28.303201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13d0bc-f200-4940-a81c-a6401c216dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:28.303201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13d0bc-f200-4940-a81c-a6401c216dd4 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:28.304515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 649/1293] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:28.310925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c7a7-f052-4779-b6a7-e96181dcc38f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:28.311637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c7a7-f052-4779-b6a7-e96181dcc38f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:28.311780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04c7a7-f052-4779-b6a7-e96181dcc38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:28.339080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cf04c7a7-f052-4779-b6a7-e96181dcc38f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:28.340423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cf04c7a7-f052-4779-b6a7-e96181dcc38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:28.370352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:28.412179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 645/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:28.443638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13d0bc-f200-4940-a81c-a6401c216dd4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:28.483648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:28.563419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13d0bc-f200-4940-a81c-a6401c216dd4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:28.612005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:28.612664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:28.612944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:28.613216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:28.613956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:28.615353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85ea6a-7548-4c4a-897c-c03c88df0262 None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:28.633167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 650/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Mon Jul 4 04:05:28 2022] POST /identity/v3/services => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:05:30.623746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e32e32-b0b9-44f6-b8ca-27e34c94ba0f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:30.624523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e32e32-b0b9-44f6-b8ca-27e34c94ba0f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:30.624523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e32e32-b0b9-44f6-b8ca-27e34c94ba0f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:30.626654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 646/1296] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:30.634032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:30.634780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:30.636442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:30.663740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:30.665589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:30.752809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 651/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:30.779231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e32e32-b0b9-44f6-b8ca-27e34c94ba0f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:30.890809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e32e32-b0b9-44f6-b8ca-27e34c94ba0f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:30.928521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:30.929314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:30.929596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:30.929873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:30.930581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:30.931983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:30.944546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:30.945697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:30.947040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 647/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:05:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:30.952569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:30.971498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:31.057757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b58ae-43c4-422c-b3c1-ff74a9841d8e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:31.093756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:31.094416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:31.094722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:31.094997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:31.095712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:31.096018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:31.097386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:31.102654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 652/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Jul 4 04:05:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:05:31.108792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:31.141713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:31.221055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34793f37-7c38-4b46-9aa9-40dbc8e6c8d4 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:31.256326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:31.257282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:31.257559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:31.257830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:31.258648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] RBAC: Adding query filter params (interface=public, service_id=efe4bd47c2d343dd8eabdb1e4cae6f6a, region_id=RegionOne) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:31.258957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:31.260359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec63bcf-5ecf-40e0-ba53-459c2856997d None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:31.265491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 648/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Mon Jul 4 04:05:31 2022] GET /identity/v3/endpoints?service_id=efe4bd47c2d343dd8eabdb1e4cae6f6a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:31.274702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:31.362006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:31.386064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c59186-ded6-4594-b95b-157d7f8baaae None admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:31.424638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:31.425251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:31.425532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:31.425803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:31.426513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:31.427901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1b520-065f-4abc-a8e6-f5a03559bb2c None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:31.435131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 653/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Mon Jul 4 04:05:31 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:05:33.417794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429b4b2-5ce9-4803-9808-f2e35902df8c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:33.418411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429b4b2-5ce9-4803-9808-f2e35902df8c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:33.418411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429b4b2-5ce9-4803-9808-f2e35902df8c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:33.419752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 649/1302] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:05:33.425673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:33.426414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:33.426580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:33.446261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:05:33.447519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:05:33.517197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 654/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:05:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 04:05:33.540450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429b4b2-5ce9-4803-9808-f2e35902df8c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:33.660221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:33.669008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429b4b2-5ce9-4803-9808-f2e35902df8c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:33.710170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:33.711018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:33.711302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:33.711577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:33.712237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:33.713689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 04:05:33.725726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 04:05:33.726775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application with Lock( Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self._enter() Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application return self.creator() Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application created_value = creator( Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: ERROR keystone.server.flask.application Jul 04 04:05:33.727621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 650/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Mon Jul 4 04:05:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 04:05:33.733419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:33.813316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:33.838427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef918dc-99c9-4459-aa9e-e865d939cea6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:33.873694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:33.874351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:33.874661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:33.874938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] PATH_INFO: `/v3/services` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:33.875651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:05:33.875958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:33.877316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1a415-2415-4536-afa3-6d6ea201fbfc None admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:33.882922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 655/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Mon Jul 4 04:05:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:05:33.889496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:05:33.996543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f1e21-3270-4742-8584-917123cc0378 None admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:05:34.032676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:05:34.033312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:05:34.033594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:05:34.033868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:05:34.034743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:05:34.036113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f890e90-9a19-4b63-b71e-bcb73fa0e348 None admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:05:34.060065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 651/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Mon Jul 4 04:05:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:05:43.880650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=30 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:05:44.059593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:37.022221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0a4a-46b7-4b88-83cf-e1661d609c2e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:37.023349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0a4a-46b7-4b88-83cf-e1661d609c2e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:37.023938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0a4a-46b7-4b88-83cf-e1661d609c2e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:37.029483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 656/1307] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:37.031039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a321-da47-43a5-9792-a8a28e947fdb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:37.031476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a321-da47-43a5-9792-a8a28e947fdb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:37.031649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a321-da47-43a5-9792-a8a28e947fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:37.105617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4547a321-da47-43a5-9792-a8a28e947fdb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:37.106792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4547a321-da47-43a5-9792-a8a28e947fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:37.242862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 652/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:06:37.279192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:37.279760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:37.280187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:37.282433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 657/1309] 10.222.0.16 () {54 vars in 964 bytes} [Mon Jul 4 04:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:37.289710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c191-08c9-420a-b88e-15d6b0cfb46c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:37.290085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c191-08c9-420a-b88e-15d6b0cfb46c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:37.290409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c191-08c9-420a-b88e-15d6b0cfb46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:37.410525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2a35c191-08c9-420a-b88e-15d6b0cfb46c None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:37.412289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2a35c191-08c9-420a-b88e-15d6b0cfb46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:37.439127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:37.525348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 653/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Jul 4 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:06:37.534327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:37.536088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:37.661633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a77f0-342f-4454-8eac-529e367aa84a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:37.742749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:37.743430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:37.743704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:37.743964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:37.745613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:37.803201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:37.864118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:37.865576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d81792-96e5-483a-adcb-ad8e81cc1df2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:37.886531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 658/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:06:40.180917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:40.181453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:40.181453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:40.182938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 654/1312] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:40.188479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c038ca-0afa-4834-8370-ea73a7638716 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:40.188975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c038ca-0afa-4834-8370-ea73a7638716 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:40.189057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c038ca-0afa-4834-8370-ea73a7638716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:40.277693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b5c038ca-0afa-4834-8370-ea73a7638716 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:40.278938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b5c038ca-0afa-4834-8370-ea73a7638716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:40.410790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 659/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:40.450097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:40.452028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:40.570209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9d41e-8ccd-4869-ad32-d7f507eef8be None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:40.657151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:40.660123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:40.660805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:40.661082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:40.661335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:40.663926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:40.794495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:40.795961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd217c6-dd54-4ed0-bf13-68029f3bf529 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:40.814911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 655/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:06:43.166243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:43.166839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:43.166839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:43.168221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 660/1315] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:43.173540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7618-19ba-45e3-86eb-5930f894577e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:43.173992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7618-19ba-45e3-86eb-5930f894577e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:43.174086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7618-19ba-45e3-86eb-5930f894577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:43.239290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-420d7618-19ba-45e3-86eb-5930f894577e None None] MFA Rules not processed for user `35b22ca699494e3baf04c55a51b0ee59`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:43.240480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-420d7618-19ba-45e3-86eb-5930f894577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:43.294303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:43.340541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 656/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:43.383804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:43.386231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:43.401418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:43.503218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848907e4-7b8a-46c9-bd61-be207eff9d9f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:43.583728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:43.584397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:43.584680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:43.584953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:43.586756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:43.664959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:43.715413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:43.716869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deb6a5-4d41-45ff-bcc5-c29420dccfa1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:43.736251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 661/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:06:46.283467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:46.284030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:46.284030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:46.285423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 657/1318] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:46.290897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8066-9042-419c-88d7-ea07956ad74e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:46.291336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8066-9042-419c-88d7-ea07956ad74e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:46.291426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8066-9042-419c-88d7-ea07956ad74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:46.358268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2fef8066-9042-419c-88d7-ea07956ad74e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:46.359497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2fef8066-9042-419c-88d7-ea07956ad74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:46.465685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 662/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:46.504383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:46.506271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:46.611676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331fb2d-9c34-4333-816a-4e0835c620af None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:46.691634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:46.692288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:46.692567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:46.692852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:46.694508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:46.701466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:46.807934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:46.809379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d718-2bec-434a-aed4-3910b74806a2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:46.827528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 658/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:06:49.519679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:49.520283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:49.520283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:49.521667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 663/1321] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:49.527075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995342-389f-4303-a114-bc48069e8471 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:49.527561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995342-389f-4303-a114-bc48069e8471 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:49.527667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995342-389f-4303-a114-bc48069e8471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:49.594110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2e995342-389f-4303-a114-bc48069e8471 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:49.595367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2e995342-389f-4303-a114-bc48069e8471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:49.668652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:49.716559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 659/1322] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:49.753557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:49.755476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:49.777049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:49.866747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16ce2e-6158-4a3a-b778-9667bdfd6301 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:49.943321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:49.943977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:49.944265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:49.944539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:49.946470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:50.022664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:50.062853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:50.064212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd0480-0509-4e36-84fd-a3127506929c service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:50.082864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 664/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:06:52.153227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045474c-fca7-4c2e-b1af-2c05513ec4d1 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:52.153758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045474c-fca7-4c2e-b1af-2c05513ec4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:52.153758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045474c-fca7-4c2e-b1af-2c05513ec4d1 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:52.155250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 660/1324] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:52.160736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f9762-08a8-4062-b64b-c58e21fd8062 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:52.161128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f9762-08a8-4062-b64b-c58e21fd8062 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:52.161297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f9762-08a8-4062-b64b-c58e21fd8062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:52.228619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ac8f9762-08a8-4062-b64b-c58e21fd8062 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:52.229823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ac8f9762-08a8-4062-b64b-c58e21fd8062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:52.329705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 665/1325] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:52.377817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:52.378198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:52.378568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:52.380343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 661/1326] 10.222.0.16 () {54 vars in 964 bytes} [Mon Jul 4 04:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:52.386588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf737fe-9d42-4294-a444-129850068d67 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:52.386876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf737fe-9d42-4294-a444-129850068d67 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:52.387154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf737fe-9d42-4294-a444-129850068d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:52.418526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:52.464352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0bf737fe-9d42-4294-a444-129850068d67 None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:52.465623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0bf737fe-9d42-4294-a444-129850068d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:52.549280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 666/1327] 10.222.0.16 () {58 vars in 1061 bytes} [Mon Jul 4 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:52.559142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:52.560676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:52.667129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2316b-1031-481d-a9a1-f74b96d1b1d4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:52.760494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:52.761608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:52.762024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:52.762302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:52.764101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:52.784523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:52.884346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:52.885802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34060e62-bf1d-4910-8ede-0f3ebfc89d9e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:52.905384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 662/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:06:55.202957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:55.203536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:55.203536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:55.204878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 667/1329] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:55.210250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915685-21bd-4df3-ba4a-1e7ab9473f96 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:55.210949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915685-21bd-4df3-ba4a-1e7ab9473f96 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:55.210949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915685-21bd-4df3-ba4a-1e7ab9473f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:55.277042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-73915685-21bd-4df3-ba4a-1e7ab9473f96 None None] MFA Rules not processed for user `35b22ca699494e3baf04c55a51b0ee59`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:55.278257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-73915685-21bd-4df3-ba4a-1e7ab9473f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:55.358418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:55.398932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 663/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:55.436373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:55.437975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:55.552424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec34f-079f-41c1-a3be-83373d713112 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:55.578979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:55.637029 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:55.637688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:55.637969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:55.638246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:55.639994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:55.760671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:55.762098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc44c4-5169-4bc7-ac9a-b9e2b5617e44 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:55.780478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 668/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:06:58.217794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:58.218708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:58.218708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:58.220964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 664/1332] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:06:58.229841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5223e-294e-4e4d-8417-22b325ed1436 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:58.230638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5223e-294e-4e4d-8417-22b325ed1436 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:58.230796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5223e-294e-4e4d-8417-22b325ed1436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:58.301172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:58.304792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0ee5223e-294e-4e4d-8417-22b325ed1436 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:06:58.306818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0ee5223e-294e-4e4d-8417-22b325ed1436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:58.413142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 669/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:06:58.462824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:06:58.465244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:58.582590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137a9ac-97fa-4737-9a1f-91d09dbb77b4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:06:58.667378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:06:58.668036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:06:58.668320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:06:58.668587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:06:58.670224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:06:58.698352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:06:58.786110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:06:58.787611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42e98-e872-4c3f-8633-cfd6023ecf39 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:06:58.806253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 665/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:00.868117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:00.868701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:00.868701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:00.870820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 670/1335] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:00.878721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d8eb-f6b2-43b7-a73c-effdf16ace21 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:00.879285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d8eb-f6b2-43b7-a73c-effdf16ace21 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:00.879285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4d8eb-f6b2-43b7-a73c-effdf16ace21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:00.951659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-82b4d8eb-f6b2-43b7-a73c-effdf16ace21 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:00.952777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-82b4d8eb-f6b2-43b7-a73c-effdf16ace21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:01.032362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:01.055658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 666/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:07:01.098855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:01.101386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:01.220478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cd612-6dcf-4a1f-aa12-d8bd01e61d42 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:01.248759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:01.304367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:01.305021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:01.305302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:01.305576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:01.307261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:01.421256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:01.422714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd98691-3ba9-48d8-a210-0e03ea97e32e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:01.440809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 671/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:03.786142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:03.787034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:03.787263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:03.789008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 667/1338] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:03.794708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9710-b649-4eeb-8b52-f069182e1e65 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:03.795149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9710-b649-4eeb-8b52-f069182e1e65 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:03.795239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9710-b649-4eeb-8b52-f069182e1e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:03.863435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f8ad9710-b649-4eeb-8b52-f069182e1e65 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:03.864670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8ad9710-b649-4eeb-8b52-f069182e1e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:03.873209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:03.968167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 672/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:07:04.007954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:04.009468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:04.118234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f806de-799e-4ae3-93c5-e45b829be529 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:04.194047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:04.194788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:04.195065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:04.195339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:04.196985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:04.236606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:04.314532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:04.315978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199becd-e4a1-48e3-a4c5-26b6d9fc05d2 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:04.334744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 668/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:07:08.514261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:08.514818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:08.514818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:08.516115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 673/1341] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:08.521483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0601ac-58d1-40d6-b6d6-fe45ca44f191 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:08.521924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0601ac-58d1-40d6-b6d6-fe45ca44f191 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:08.522024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0601ac-58d1-40d6-b6d6-fe45ca44f191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:08.599049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ef0601ac-58d1-40d6-b6d6-fe45ca44f191 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:08.600323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ef0601ac-58d1-40d6-b6d6-fe45ca44f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:08.699356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:08.713795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 669/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:08.752691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:08.755229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:08.861608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96fe5-e396-43f4-971a-b069a1d65507 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:08.865493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:08.866531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:08.866815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:08.867091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:08.876520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:08.877942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4904808-6031-44bc-aa90-6f4363fcbac7 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:08.888477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 674/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:11.554240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:11.554836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:11.554836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:11.556233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 670/1344] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:11.561714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895943-a588-4e4f-a92b-e111895a17c1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:11.562163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895943-a588-4e4f-a92b-e111895a17c1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:11.562405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895943-a588-4e4f-a92b-e111895a17c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:11.598318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-84895943-a588-4e4f-a92b-e111895a17c1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:11.599597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-84895943-a588-4e4f-a92b-e111895a17c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:11.639815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:11.692194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 675/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:11.729960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:11.732199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:11.747056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde173-80b3-49ee-b3e7-1592443b35cd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:11.750937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:11.751603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:11.751894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:11.752167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:11.764388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:11.765979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8991c-5363-4651-8cc2-06d74de67451 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:11.776401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 671/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:16.088681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b3908-b38b-4902-a40a-ad09e062f61f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:16.089276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b3908-b38b-4902-a40a-ad09e062f61f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:16.089276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b3908-b38b-4902-a40a-ad09e062f61f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:16.091228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 676/1347] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:16.096221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1766c3c-fc6c-48db-93b8-d526760af362 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:16.096728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1766c3c-fc6c-48db-93b8-d526760af362 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:16.096728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1766c3c-fc6c-48db-93b8-d526760af362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:16.114182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d1766c3c-fc6c-48db-93b8-d526760af362 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:16.115386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d1766c3c-fc6c-48db-93b8-d526760af362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:16.123505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 672/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:16.161393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b3908-b38b-4902-a40a-ad09e062f61f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:16.173061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b3908-b38b-4902-a40a-ad09e062f61f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:16.177247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:16.177899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:16.178748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:16.178748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:16.188035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:16.189471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5dce9-454a-492f-9ef3-bd521452c655 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:16.198914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 677/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:21.123721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18448a31-b36d-467c-8087-7a2e5e29bfd6 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:21.124581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18448a31-b36d-467c-8087-7a2e5e29bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:21.125091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18448a31-b36d-467c-8087-7a2e5e29bfd6 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:21.126907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 673/1350] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:21.133651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279ae0-4dc9-466e-a326-44fda824dd02 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:21.134865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279ae0-4dc9-466e-a326-44fda824dd02 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:21.135383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279ae0-4dc9-466e-a326-44fda824dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:21.259337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3d279ae0-4dc9-466e-a326-44fda824dd02 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:21.260513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3d279ae0-4dc9-466e-a326-44fda824dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:21.342251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 678/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:07:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:25.375715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f821e5-fe1f-47e6-be02-0e2146ecbc38 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:25.378503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f821e5-fe1f-47e6-be02-0e2146ecbc38 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:25.378503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f821e5-fe1f-47e6-be02-0e2146ecbc38 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:25.378503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 674/1352] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:25.385723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b653-9074-46dd-8b92-bef01295e4c9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:25.386187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b653-9074-46dd-8b92-bef01295e4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:25.386283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b653-9074-46dd-8b92-bef01295e4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:25.405287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-afd5b653-9074-46dd-8b92-bef01295e4c9 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:25.406588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-afd5b653-9074-46dd-8b92-bef01295e4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:25.415790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 679/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:07:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:26.110836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=13 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:27.141163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ad51-3580-4df7-be94-d7029e876d5f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:27.141860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ad51-3580-4df7-be94-d7029e876d5f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:27.141860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ad51-3580-4df7-be94-d7029e876d5f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:27.143113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 675/1354] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:27.148539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be065-02ec-4211-93a1-1f58f25ca72b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:27.148996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be065-02ec-4211-93a1-1f58f25ca72b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:27.149107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be065-02ec-4211-93a1-1f58f25ca72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:27.167773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b89be065-02ec-4211-93a1-1f58f25ca72b None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:27.168984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b89be065-02ec-4211-93a1-1f58f25ca72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:27.178792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 680/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:07:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:30.731914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8c6f-345d-4605-80b2-0faeb7f05258 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:30.732414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8c6f-345d-4605-80b2-0faeb7f05258 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:30.732414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8c6f-345d-4605-80b2-0faeb7f05258 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:30.733925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 676/1356] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:30.739167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563c79-23e8-4b39-a919-611457d8ad3a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:30.739593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563c79-23e8-4b39-a919-611457d8ad3a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:30.739728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4563c79-23e8-4b39-a919-611457d8ad3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:30.762577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e4563c79-23e8-4b39-a919-611457d8ad3a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:30.763780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e4563c79-23e8-4b39-a919-611457d8ad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:30.773496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 681/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:30.829159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb718-f185-4c54-8b67-337da4c7807b None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:30.829950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb718-f185-4c54-8b67-337da4c7807b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:30.830415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb718-f185-4c54-8b67-337da4c7807b None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:30.832945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 677/1358] 10.222.0.16 () {54 vars in 961 bytes} [Mon Jul 4 04:07:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:30.840698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a362-9b77-4363-9260-78e3623a533f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:30.841060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a362-9b77-4363-9260-78e3623a533f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:30.841411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a362-9b77-4363-9260-78e3623a533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:30.886174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:30.894959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0766a362-9b77-4363-9260-78e3623a533f None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:30.896302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0766a362-9b77-4363-9260-78e3623a533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:30.988197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 682/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Jul 4 04:07:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:30.996488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cb718-f185-4c54-8b67-337da4c7807b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:31.011518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cb718-f185-4c54-8b67-337da4c7807b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:31.016988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:31.017850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:31.018138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:31.018451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:31.027457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:31.029052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37316ad6-a661-464f-855e-559a8d76f7b3 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:31.039099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 678/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:33.248676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3910-9104-412d-8cae-d0aa36220a68 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:33.249604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3910-9104-412d-8cae-d0aa36220a68 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:33.250124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3910-9104-412d-8cae-d0aa36220a68 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:33.251690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 683/1361] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:33.257594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a557a-a5a8-45ba-a05d-90b5cba393d3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:33.258013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a557a-a5a8-45ba-a05d-90b5cba393d3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:33.258143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a557a-a5a8-45ba-a05d-90b5cba393d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:33.277403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-937a557a-a5a8-45ba-a05d-90b5cba393d3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:33.278854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-937a557a-a5a8-45ba-a05d-90b5cba393d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:33.287764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 679/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:33.332058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea3910-9104-412d-8cae-d0aa36220a68 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:33.348605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea3910-9104-412d-8cae-d0aa36220a68 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:33.354996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:33.355843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:33.356279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:33.356708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:33.365802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:33.367637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f7e38-7776-4031-b044-18880c8c0606 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:33.377761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 684/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:35.736772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90903751-5a02-437a-84fd-c9789b327a7d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:35.737766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90903751-5a02-437a-84fd-c9789b327a7d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:35.738247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90903751-5a02-437a-84fd-c9789b327a7d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:35.740293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 680/1364] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:35.746427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0015c3-5ab3-41c8-9241-f4ef39267913 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:35.747380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0015c3-5ab3-41c8-9241-f4ef39267913 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:35.747884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0015c3-5ab3-41c8-9241-f4ef39267913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:35.767585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6f0015c3-5ab3-41c8-9241-f4ef39267913 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:35.769079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6f0015c3-5ab3-41c8-9241-f4ef39267913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:35.777855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 685/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:35.822870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade829-e95e-4900-b640-5aab49c7797a None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:35.823162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade829-e95e-4900-b640-5aab49c7797a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:35.823467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade829-e95e-4900-b640-5aab49c7797a None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:35.824885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 681/1366] 10.222.0.16 () {54 vars in 961 bytes} [Mon Jul 4 04:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:35.830279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed78c-0597-4ece-a9ce-1d6c39c5e29c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:35.830614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed78c-0597-4ece-a9ce-1d6c39c5e29c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:35.830896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed78c-0597-4ece-a9ce-1d6c39c5e29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:35.849895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-189ed78c-0597-4ece-a9ce-1d6c39c5e29c None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:35.851129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-189ed78c-0597-4ece-a9ce-1d6c39c5e29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:35.860501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 686/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Jul 4 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:35.868631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ade829-e95e-4900-b640-5aab49c7797a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:35.885093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ade829-e95e-4900-b640-5aab49c7797a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:35.890970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:35.891612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:35.891897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:35.892189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:35.900845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:35.902472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94070d-d626-4e57-bd85-5de199a7bf7a service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:35.912129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 682/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:38.084646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3835-8ffd-4055-a34e-30296081aefa None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:38.085225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3835-8ffd-4055-a34e-30296081aefa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:38.085225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3835-8ffd-4055-a34e-30296081aefa None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:38.086646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 687/1369] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:38.092705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec80d-1c73-48c9-bcba-1d671376f74a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:38.093320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec80d-1c73-48c9-bcba-1d671376f74a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:38.093320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec80d-1c73-48c9-bcba-1d671376f74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:38.112559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-80eec80d-1c73-48c9-bcba-1d671376f74a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:38.113760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-80eec80d-1c73-48c9-bcba-1d671376f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:38.122832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 683/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:38.167146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3835-8ffd-4055-a34e-30296081aefa None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:38.182024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3835-8ffd-4055-a34e-30296081aefa None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:38.188555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:38.189488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:38.189927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:38.190348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:38.200049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:38.201836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dad9ab-13be-414e-9851-751681c3c7db service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:38.213149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 688/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:38.743880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd13c5-2ff9-45aa-b204-3d8f5d8275f9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:38.744114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd13c5-2ff9-45aa-b204-3d8f5d8275f9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:38.744428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd13c5-2ff9-45aa-b204-3d8f5d8275f9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:38.745789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 684/1372] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:38.751302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0cf3-2a1e-4ce7-ae84-def003baa1a5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:38.751547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0cf3-2a1e-4ce7-ae84-def003baa1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:38.752000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0cf3-2a1e-4ce7-ae84-def003baa1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:38.773110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d52c0cf3-2a1e-4ce7-ae84-def003baa1a5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:38.774319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d52c0cf3-2a1e-4ce7-ae84-def003baa1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:38.783027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 689/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:39.135309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f91981-4895-45d4-818e-ace2f3df70d3 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:39.135309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f91981-4895-45d4-818e-ace2f3df70d3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:39.135309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f91981-4895-45d4-818e-ace2f3df70d3 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:39.137786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 685/1374] 10.222.0.16 () {54 vars in 963 bytes} [Mon Jul 4 04:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:39.141792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c17c9-eb1a-4047-98c4-a29ea5c4e915 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:39.142250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c17c9-eb1a-4047-98c4-a29ea5c4e915 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:39.142422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c17c9-eb1a-4047-98c4-a29ea5c4e915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:39.196603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8e9c17c9-eb1a-4047-98c4-a29ea5c4e915 None None] MFA Rules not processed for user `0f9c5095b2ac44c1833141d0949c6ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:39.197804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8e9c17c9-eb1a-4047-98c4-a29ea5c4e915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:39.262536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 690/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Mon Jul 4 04:07:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:39.267944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f91981-4895-45d4-818e-ace2f3df70d3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:39.278946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f91981-4895-45d4-818e-ace2f3df70d3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:39.282568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:39.283191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:39.283666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:39.283941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:39.293838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:39.295301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3aca60-f0fb-4175-b9c5-12259c96094f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:39.305037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 686/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:40.294442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763769-97ca-429d-a59a-0279144baa8e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:40.295367 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763769-97ca-429d-a59a-0279144baa8e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:40.295649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763769-97ca-429d-a59a-0279144baa8e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:40.300955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 691/1377] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:40.302590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79405c1f-b76a-4404-bc4a-6a7e7626ba29 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:40.303152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79405c1f-b76a-4404-bc4a-6a7e7626ba29 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:40.303152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79405c1f-b76a-4404-bc4a-6a7e7626ba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:40.321375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-79405c1f-b76a-4404-bc4a-6a7e7626ba29 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:40.322640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-79405c1f-b76a-4404-bc4a-6a7e7626ba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:40.331545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 687/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:40.375880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd763769-97ca-429d-a59a-0279144baa8e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:40.392705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd763769-97ca-429d-a59a-0279144baa8e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:40.398818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:40.399481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:40.399768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:40.400048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:40.408598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:40.410506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64203-329c-4a97-914d-640ce14c4929 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:40.414851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:40.421840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 692/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:42.464524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ca6c-95b7-49fa-b92c-e9da932797e0 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:42.465089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ca6c-95b7-49fa-b92c-e9da932797e0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:42.465089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ca6c-95b7-49fa-b92c-e9da932797e0 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:42.466484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 688/1380] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:42.471731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db287a-0c75-4ddd-bd4e-7746a0723f45 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:42.472161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db287a-0c75-4ddd-bd4e-7746a0723f45 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:42.472263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db287a-0c75-4ddd-bd4e-7746a0723f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:42.490514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-21db287a-0c75-4ddd-bd4e-7746a0723f45 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:42.491669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-21db287a-0c75-4ddd-bd4e-7746a0723f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:42.500373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 693/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:42.544550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942ca6c-95b7-49fa-b92c-e9da932797e0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:42.559880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942ca6c-95b7-49fa-b92c-e9da932797e0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:42.568963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:42.570028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:42.570577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:42.571123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:42.580670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:42.582347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004094ea-355a-41cd-bfd7-d2e59c158406 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:42.592030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 689/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:44.672309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b66be-032a-48f2-bc56-dbc87a45e9fc None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:44.672967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b66be-032a-48f2-bc56-dbc87a45e9fc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:44.672967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b66be-032a-48f2-bc56-dbc87a45e9fc None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:44.674213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 694/1383] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:44.679951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44cfe-9dab-4308-9b82-5daedb1d2155 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:44.680793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44cfe-9dab-4308-9b82-5daedb1d2155 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:44.681301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44cfe-9dab-4308-9b82-5daedb1d2155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:44.701341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-73e44cfe-9dab-4308-9b82-5daedb1d2155 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:44.702900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-73e44cfe-9dab-4308-9b82-5daedb1d2155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:44.712511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 690/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:44.754840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b66be-032a-48f2-bc56-dbc87a45e9fc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:44.773151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b66be-032a-48f2-bc56-dbc87a45e9fc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:44.779544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:44.780020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:44.780345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:44.780798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:44.790178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:44.792052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4e04b-440a-495d-bb6c-da852e3603d6 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:44.802668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 695/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:46.834748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398318-f12a-4a52-a9a1-98231614a21d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:46.835277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398318-f12a-4a52-a9a1-98231614a21d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:46.835277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70398318-f12a-4a52-a9a1-98231614a21d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:46.836748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 691/1386] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:07:46.842223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588a6e-8711-4a79-ba40-b09dcce5391c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:46.842686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588a6e-8711-4a79-ba40-b09dcce5391c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:46.842768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588a6e-8711-4a79-ba40-b09dcce5391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:46.862299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-37588a6e-8711-4a79-ba40-b09dcce5391c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:46.863764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-37588a6e-8711-4a79-ba40-b09dcce5391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:46.872961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 696/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:46.929756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70398318-f12a-4a52-a9a1-98231614a21d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:46.947353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70398318-f12a-4a52-a9a1-98231614a21d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:46.953633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:46.954459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:46.954764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:46.955131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:46.966261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:46.968262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a6628-2d3a-45d7-b5a2-da392ee2bb3c service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:46.980181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 692/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:49.156135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b7f2-6d18-48fb-a2cd-e264eff78f07 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:49.156894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b7f2-6d18-48fb-a2cd-e264eff78f07 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:49.156894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b7f2-6d18-48fb-a2cd-e264eff78f07 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:49.158208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 697/1389] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:49.164768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04fa4-8169-43fa-9e1a-9831d728a1e3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:49.165244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04fa4-8169-43fa-9e1a-9831d728a1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:49.165649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04fa4-8169-43fa-9e1a-9831d728a1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:49.187690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b8f04fa4-8169-43fa-9e1a-9831d728a1e3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:49.188930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b8f04fa4-8169-43fa-9e1a-9831d728a1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:49.198294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 693/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:49.243979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456b7f2-6d18-48fb-a2cd-e264eff78f07 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:49.261377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456b7f2-6d18-48fb-a2cd-e264eff78f07 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:49.269750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:49.270955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:49.271332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:49.271748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:49.285069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:49.287364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd43a17-88b2-4bda-8ae8-b9bd7604c783 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:49.297306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 698/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:51.420616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69677-f71c-439a-8101-be5a62f2c3cf None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:51.421233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69677-f71c-439a-8101-be5a62f2c3cf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:51.421233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69677-f71c-439a-8101-be5a62f2c3cf None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:51.422689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 694/1392] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:51.428723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddfbf-bb05-4e5b-8a6e-bbf3faba2db0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:51.429211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddfbf-bb05-4e5b-8a6e-bbf3faba2db0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:51.429375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddfbf-bb05-4e5b-8a6e-bbf3faba2db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:51.452198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-881ddfbf-bb05-4e5b-8a6e-bbf3faba2db0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:51.453742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-881ddfbf-bb05-4e5b-8a6e-bbf3faba2db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:51.464572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 699/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:51.516140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b69677-f71c-439a-8101-be5a62f2c3cf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:51.533126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b69677-f71c-439a-8101-be5a62f2c3cf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:51.539682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:51.540367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:51.540694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:51.540979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:51.550437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:51.552260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d4360-541d-4e7e-a745-efb2ed478d6a service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:51.562842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 695/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:53.656751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a39ec-98c6-45e4-b2ff-cdde15914126 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:53.657326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a39ec-98c6-45e4-b2ff-cdde15914126 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:53.657326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a39ec-98c6-45e4-b2ff-cdde15914126 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:53.658948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 700/1395] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:53.664301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c6ed-b613-4a95-9104-de4b6a45aa3c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:53.664737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c6ed-b613-4a95-9104-de4b6a45aa3c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:53.664901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c6ed-b613-4a95-9104-de4b6a45aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:53.684027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-49a9c6ed-b613-4a95-9104-de4b6a45aa3c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:53.685245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-49a9c6ed-b613-4a95-9104-de4b6a45aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:53.694353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 696/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:53.738822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a39ec-98c6-45e4-b2ff-cdde15914126 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:53.755903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a39ec-98c6-45e4-b2ff-cdde15914126 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:53.761706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:53.762396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:53.762709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:53.762985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:53.771712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:53.773302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88a1f9-6368-4b85-b805-f453045e7770 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:53.783016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 701/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:54.339699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52969d-a82d-451a-a6a4-3eb99d475aba None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:54.340001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52969d-a82d-451a-a6a4-3eb99d475aba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:54.340301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52969d-a82d-451a-a6a4-3eb99d475aba None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:54.341705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 697/1398] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:54.347022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ac46-a963-4a32-83d1-d3f88d6c7cbc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:54.347329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ac46-a963-4a32-83d1-d3f88d6c7cbc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:54.347607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ac46-a963-4a32-83d1-d3f88d6c7cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:54.365844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b6e9ac46-a963-4a32-83d1-d3f88d6c7cbc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:54.367070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b6e9ac46-a963-4a32-83d1-d3f88d6c7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:54.375437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 702/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:54.431334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1378e5-a5ad-4080-98c5-652facea8013 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:54.431622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1378e5-a5ad-4080-98c5-652facea8013 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:54.431897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1378e5-a5ad-4080-98c5-652facea8013 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:54.433274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 698/1400] 10.222.0.16 () {54 vars in 962 bytes} [Mon Jul 4 04:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:54.438548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a504-93cb-45ae-9dd0-0915c057b412 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:54.438843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a504-93cb-45ae-9dd0-0915c057b412 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:54.439163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a504-93cb-45ae-9dd0-0915c057b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:54.488347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-54a1a504-93cb-45ae-9dd0-0915c057b412 None None] MFA Rules not processed for user `dcab788fe9b0499c99336de1c06c885d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:54.489631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-54a1a504-93cb-45ae-9dd0-0915c057b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:54.552102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 703/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Jul 4 04:07:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:54.559927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1378e5-a5ad-4080-98c5-652facea8013 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:54.570976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1378e5-a5ad-4080-98c5-652facea8013 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:54.574507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:54.575135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:54.575414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:54.575673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:54.584573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:54.586006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a76af-0fc6-4b0d-a4d0-9291550f419e service glance] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:54.595271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 699/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:07:55.309173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd06185-d032-4e9d-ba81-07703cbcdb75 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.309647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd06185-d032-4e9d-ba81-07703cbcdb75 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.309739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd06185-d032-4e9d-ba81-07703cbcdb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.358169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ffd06185-d032-4e9d-ba81-07703cbcdb75 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:55.359405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ffd06185-d032-4e9d-ba81-07703cbcdb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:55.400684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:07:55.427500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 704/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:07:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:55.472517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd509-0cb6-43ef-98e5-4fadb4ac9ce7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.472988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd509-0cb6-43ef-98e5-4fadb4ac9ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.473162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd509-0cb6-43ef-98e5-4fadb4ac9ce7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.475266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 700/1404] 10.222.0.16 () {54 vars in 942 bytes} [Mon Jul 4 04:07:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:55.481785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4022ec-a3c3-4472-b54c-ef56afb0d6db None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.482079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4022ec-a3c3-4472-b54c-ef56afb0d6db None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.482358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4022ec-a3c3-4472-b54c-ef56afb0d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.534202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9c4022ec-a3c3-4472-b54c-ef56afb0d6db None None] MFA Rules not processed for user `01612217d9fb486daccdc549d9e0c6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:55.535462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9c4022ec-a3c3-4472-b54c-ef56afb0d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:55.609021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 705/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Mon Jul 4 04:07:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:55.618553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cd509-0cb6-43ef-98e5-4fadb4ac9ce7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:55.635067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cd509-0cb6-43ef-98e5-4fadb4ac9ce7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:55.640133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:55.640785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.641063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.641339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.649853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:55.651448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648dcf4-5fdb-4090-bc67-d4b38a24e0ce service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:55.659563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 701/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:07:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:07:55.843077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d143fb-e57f-4195-a2b9-53f4c61584c0 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.843275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d143fb-e57f-4195-a2b9-53f4c61584c0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.843528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d143fb-e57f-4195-a2b9-53f4c61584c0 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.844931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 706/1407] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:07:55.850355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc4ae-cc71-4ace-ba84-cd840da72e2e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.850682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc4ae-cc71-4ace-ba84-cd840da72e2e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.850965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc4ae-cc71-4ace-ba84-cd840da72e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.870947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-218fc4ae-cc71-4ace-ba84-cd840da72e2e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:07:55.872500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-218fc4ae-cc71-4ace-ba84-cd840da72e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:07:55.882145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 702/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:07:55.931088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d143fb-e57f-4195-a2b9-53f4c61584c0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:07:55.946923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d143fb-e57f-4195-a2b9-53f4c61584c0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:07:55.952642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:07:55.955173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:07:55.955173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:07:55.955173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:07:55.964028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:07:55.965639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07892-05fb-488d-afe3-b9561a6395cd service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:07:55.975844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 707/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:05.865002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=69 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:08:05.971650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=59 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:08:17.998877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5261-2c59-4b50-9f79-3529d3056ebe None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:17.999479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5261-2c59-4b50-9f79-3529d3056ebe None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:17.999479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5261-2c59-4b50-9f79-3529d3056ebe None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:18.000842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 703/1410] 10.222.0.16 () {54 vars in 918 bytes} [Mon Jul 4 04:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:18.007744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174249e-fafb-45ae-955a-5b36613f4ada None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:18.008227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174249e-fafb-45ae-955a-5b36613f4ada None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:18.008430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174249e-fafb-45ae-955a-5b36613f4ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:18.036178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3174249e-fafb-45ae-955a-5b36613f4ada None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:18.037381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3174249e-fafb-45ae-955a-5b36613f4ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:18.047734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 708/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Jul 4 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:18.069124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564efb-5e56-4bdb-8283-94927871a588 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:18.069433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564efb-5e56-4bdb-8283-94927871a588 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:18.069711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564efb-5e56-4bdb-8283-94927871a588 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:18.071123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 704/1412] 10.222.0.16 () {54 vars in 918 bytes} [Mon Jul 4 04:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:18.075987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ac1c-afb5-46bb-846e-d9a5b8c5fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:18.076280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ac1c-afb5-46bb-846e-d9a5b8c5fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:18.076559 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ac1c-afb5-46bb-846e-d9a5b8c5fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:18.095790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1ce2ac1c-afb5-46bb-846e-d9a5b8c5fcc2 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:18.097003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1ce2ac1c-afb5-46bb-846e-d9a5b8c5fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:18.108513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 709/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Jul 4 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:20.610042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb090-dc9f-428e-ba2a-c2a4c008a9e0 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:20.610701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb090-dc9f-428e-ba2a-c2a4c008a9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:20.610701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfb090-dc9f-428e-ba2a-c2a4c008a9e0 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:20.611998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:20.618443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b999c-23b1-4443-8c7c-667711a25c14 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:20.618885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b999c-23b1-4443-8c7c-667711a25c14 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:20.618999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b999c-23b1-4443-8c7c-667711a25c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:20.643609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3e6b999c-23b1-4443-8c7c-667711a25c14 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:20.645154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3e6b999c-23b1-4443-8c7c-667711a25c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:20.653409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:20.705805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfb090-dc9f-428e-ba2a-c2a4c008a9e0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:20.723006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfb090-dc9f-428e-ba2a-c2a4c008a9e0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:20.728996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:20.729644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:20.729934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:20.730212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:20.738752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:20.740341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5b776-a3ff-4986-804b-7b195b506a2c service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:20.749642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 706/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:21.658508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668f35-2154-477b-b283-91d2dde6c552 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:21.659041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668f35-2154-477b-b283-91d2dde6c552 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:21.659041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668f35-2154-477b-b283-91d2dde6c552 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:21.660599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:08:21.668572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c8a8-729e-45ac-98dd-66bddf7f7da7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:21.669044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c8a8-729e-45ac-98dd-66bddf7f7da7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:21.669147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c8a8-729e-45ac-98dd-66bddf7f7da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:21.690153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6e85c8a8-729e-45ac-98dd-66bddf7f7da7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:21.691446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e85c8a8-729e-45ac-98dd-66bddf7f7da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:21.699922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:21.850882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c668f35-2154-477b-b283-91d2dde6c552 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:21.867292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c668f35-2154-477b-b283-91d2dde6c552 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:21.874073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:21.874737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:21.875020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:21.875295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:21.885024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:21.886594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b53f01-b0ce-46bc-a56f-d665e19a9f32 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:21.896453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 712/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:22.791980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a977-64ff-4572-8423-40223da3d1e7 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:22.792437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a977-64ff-4572-8423-40223da3d1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:22.792576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a977-64ff-4572-8423-40223da3d1e7 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:22.794269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:22.799344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fb61c-3f87-4d7a-a72a-a07565d66101 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:22.799821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fb61c-3f87-4d7a-a72a-a07565d66101 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:22.799922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fb61c-3f87-4d7a-a72a-a07565d66101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:22.818883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4e8fb61c-3f87-4d7a-a72a-a07565d66101 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:22.820155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4e8fb61c-3f87-4d7a-a72a-a07565d66101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:22.829069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:22.986201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8a977-64ff-4572-8423-40223da3d1e7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:23.003997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8a977-64ff-4572-8423-40223da3d1e7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:23.010247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:23.010921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:23.011206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:23.011488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:23.020396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:23.022007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51b442-6228-4592-af89-86bd405aba7a service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:23.034662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 709/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:23.944607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee15b1-c0f5-40b4-80c2-98ef86c8fa38 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:23.945445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee15b1-c0f5-40b4-80c2-98ef86c8fa38 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:23.945445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee15b1-c0f5-40b4-80c2-98ef86c8fa38 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:23.947613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:23.955737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e56c0-9557-4605-b47f-bebe899cd8eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:23.956184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e56c0-9557-4605-b47f-bebe899cd8eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:23.956316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e56c0-9557-4605-b47f-bebe899cd8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:23.981117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ab5e56c0-9557-4605-b47f-bebe899cd8eb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:23.982336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ab5e56c0-9557-4605-b47f-bebe899cd8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:23.991413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:24.148497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee15b1-c0f5-40b4-80c2-98ef86c8fa38 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:24.170311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee15b1-c0f5-40b4-80c2-98ef86c8fa38 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:24.178953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:24.179979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:24.180435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:24.180837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:24.194332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:24.196022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753ae2-e105-49ed-9853-c163db8ec185 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:24.205845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 715/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:25.074317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfb5e-c7dd-4ff8-a940-df6ee9e91344 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:25.075343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfb5e-c7dd-4ff8-a940-df6ee9e91344 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:25.075343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfb5e-c7dd-4ff8-a940-df6ee9e91344 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:25.077539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:25.085626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4b62-27da-4c00-af44-d9abb2848c41 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:25.086093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4b62-27da-4c00-af44-d9abb2848c41 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:25.086199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4b62-27da-4c00-af44-d9abb2848c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:25.113750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bc1b4b62-27da-4c00-af44-d9abb2848c41 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:25.114970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bc1b4b62-27da-4c00-af44-d9abb2848c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:25.123448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:25.277966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dfb5e-c7dd-4ff8-a940-df6ee9e91344 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:25.296945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dfb5e-c7dd-4ff8-a940-df6ee9e91344 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:25.304976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:25.305643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:25.305923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:25.306169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:25.314816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:25.316445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dace805-8627-45b3-ad06-a9adbd1969b1 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:25.326752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 712/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:26.145228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725bb66-204f-492a-a65d-c8b660e364d3 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:26.145843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725bb66-204f-492a-a65d-c8b660e364d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:26.145843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725bb66-204f-492a-a65d-c8b660e364d3 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:26.147313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:26.153453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6e2c5-27ad-4fce-a823-9603c60a35e3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:26.153902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6e2c5-27ad-4fce-a823-9603c60a35e3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:26.154013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6e2c5-27ad-4fce-a823-9603c60a35e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:26.176950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-72d6e2c5-27ad-4fce-a823-9603c60a35e3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:26.178157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-72d6e2c5-27ad-4fce-a823-9603c60a35e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:26.187955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:26.351681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725bb66-204f-492a-a65d-c8b660e364d3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:26.377980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725bb66-204f-492a-a65d-c8b660e364d3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:26.385318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:26.385967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:26.386251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:26.386559 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:26.395322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:26.396842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386efe2-db7e-4474-b420-3e8d3e0976d7 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:26.406352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 718/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:27.215833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974183d-d938-4228-9c13-92ef032586ef None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:27.216380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974183d-d938-4228-9c13-92ef032586ef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:27.216380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974183d-d938-4228-9c13-92ef032586ef None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:27.218313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:27.225864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e5a1c-58a6-4c01-90fe-af3e6af8ddfd None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:27.226291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e5a1c-58a6-4c01-90fe-af3e6af8ddfd None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:27.226526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e5a1c-58a6-4c01-90fe-af3e6af8ddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:27.247428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f61e5a1c-58a6-4c01-90fe-af3e6af8ddfd None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:27.248636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f61e5a1c-58a6-4c01-90fe-af3e6af8ddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:27.260056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:27.409517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974183d-d938-4228-9c13-92ef032586ef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:27.432249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974183d-d938-4228-9c13-92ef032586ef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:27.439516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:27.440173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:27.440459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:27.440739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:27.449969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:27.451623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36562c94-42eb-443d-ac10-536cf35acd2b service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:27.463745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 715/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:28.306119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c86802-6c09-4a01-a802-382cbc162edb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:28.306783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c86802-6c09-4a01-a802-382cbc162edb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:28.306783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c86802-6c09-4a01-a802-382cbc162edb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:28.308172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:28.315184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a59e2f-ec71-40b5-abea-a06fff9a73f4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:28.315678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a59e2f-ec71-40b5-abea-a06fff9a73f4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:28.315678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a59e2f-ec71-40b5-abea-a06fff9a73f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:28.337585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-65a59e2f-ec71-40b5-abea-a06fff9a73f4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:28.338843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65a59e2f-ec71-40b5-abea-a06fff9a73f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:28.348657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:28.509881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c86802-6c09-4a01-a802-382cbc162edb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:28.527069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c86802-6c09-4a01-a802-382cbc162edb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:28.533954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:28.534640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:28.534932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:28.535301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:28.544271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:28.545894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a3f63-2fae-429a-859f-82a2114fe22d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:28.559897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 721/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:29.407477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83c4e-0b4d-4e8c-bbcf-ae92120e6767 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:29.408360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83c4e-0b4d-4e8c-bbcf-ae92120e6767 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:29.408360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83c4e-0b4d-4e8c-bbcf-ae92120e6767 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:29.410617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:08:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:29.416679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e688-54df-43bf-aa2b-a2d92c44a58a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:29.417115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e688-54df-43bf-aa2b-a2d92c44a58a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:29.417324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88e688-54df-43bf-aa2b-a2d92c44a58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:29.437434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1d88e688-54df-43bf-aa2b-a2d92c44a58a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:29.438751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1d88e688-54df-43bf-aa2b-a2d92c44a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:29.447887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:29.601021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b83c4e-0b4d-4e8c-bbcf-ae92120e6767 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:29.622787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b83c4e-0b4d-4e8c-bbcf-ae92120e6767 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:29.629589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:29.630238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:29.630559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:29.630841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:29.640684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:29.642309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c9b6c-55ac-45fc-b9cc-027a365323f6 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:29.655833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 718/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:31.594849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54761e6c-f21d-4780-bcc1-38c094afc901 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:31.595532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54761e6c-f21d-4780-bcc1-38c094afc901 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:31.595532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54761e6c-f21d-4780-bcc1-38c094afc901 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:31.597348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:31.602786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160aa7-5986-4f0b-8767-b242f5627672 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:31.603288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160aa7-5986-4f0b-8767-b242f5627672 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:31.603395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160aa7-5986-4f0b-8767-b242f5627672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:31.622568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-18160aa7-5986-4f0b-8767-b242f5627672 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:31.623781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-18160aa7-5986-4f0b-8767-b242f5627672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:31.633007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:31.675225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54761e6c-f21d-4780-bcc1-38c094afc901 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:31.692387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54761e6c-f21d-4780-bcc1-38c094afc901 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:31.696338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:31.697020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:31.697305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:31.697578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:31.707087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:31.708620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648933-fdcf-43fd-938f-b492aca6d346 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:31.718184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 724/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:33.946613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8152-81c4-4d2e-ac3c-599194d8037c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:33.947448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8152-81c4-4d2e-ac3c-599194d8037c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:33.947448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8152-81c4-4d2e-ac3c-599194d8037c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:33.949665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:33.956009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbcceb-192c-48a2-86d9-2036cfee0943 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:33.956726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbcceb-192c-48a2-86d9-2036cfee0943 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:33.956912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbcceb-192c-48a2-86d9-2036cfee0943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:33.986004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e2bbcceb-192c-48a2-86d9-2036cfee0943 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:33.987253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e2bbcceb-192c-48a2-86d9-2036cfee0943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:33.996563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:34.040639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8152-81c4-4d2e-ac3c-599194d8037c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:34.052095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8152-81c4-4d2e-ac3c-599194d8037c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:34.055893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:34.056548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:34.056832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:34.057108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:34.066040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:34.067547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dffa7-17c9-42b8-aa07-70f38cfc35cf service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:34.076912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 721/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:08:36.413577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8fc8-b745-4afd-a21c-5579632fc702 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:36.414484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8fc8-b745-4afd-a21c-5579632fc702 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:36.414484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8fc8-b745-4afd-a21c-5579632fc702 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:36.416756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:36.424843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacacff-83b8-4b47-bdab-02d9788534b2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:36.425298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacacff-83b8-4b47-bdab-02d9788534b2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:36.425391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacacff-83b8-4b47-bdab-02d9788534b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:36.443535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1aacacff-83b8-4b47-bdab-02d9788534b2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:36.444731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1aacacff-83b8-4b47-bdab-02d9788534b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:36.452951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:36.493614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8fc8-b745-4afd-a21c-5579632fc702 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:36.507785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8fc8-b745-4afd-a21c-5579632fc702 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:36.512389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:36.513032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:36.513319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:36.513598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:36.523277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:36.524718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaf80a-7f73-4922-b8a7-0288c7670107 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:36.534110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 727/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:38.685883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ea2a-6a86-4edb-8cce-300859a1a22c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:38.686481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ea2a-6a86-4edb-8cce-300859a1a22c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:38.686481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ea2a-6a86-4edb-8cce-300859a1a22c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:38.687876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:38.694583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a4d2-3f41-4b9e-ab64-c717fde83034 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:38.695363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a4d2-3f41-4b9e-ab64-c717fde83034 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:38.695533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a4d2-3f41-4b9e-ab64-c717fde83034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:38.719044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8b65a4d2-3f41-4b9e-ab64-c717fde83034 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:38.720248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8b65a4d2-3f41-4b9e-ab64-c717fde83034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:38.729759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:38.768449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ea2a-6a86-4edb-8cce-300859a1a22c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:38.781044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ea2a-6a86-4edb-8cce-300859a1a22c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:38.785162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:38.786031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:38.786511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:38.787855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:38.796805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:38.798287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5f32-06fb-4c1c-a48b-abceaf7df4c9 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:38.808217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 724/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:08:40.951884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4619-1773-43ea-892b-4d3c58165574 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:40.952463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4619-1773-43ea-892b-4d3c58165574 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:40.952463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4619-1773-43ea-892b-4d3c58165574 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:40.953891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 04:08:40.959930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec3e04-82ce-4c59-b2d6-87d93ea1a009 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:40.960414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec3e04-82ce-4c59-b2d6-87d93ea1a009 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:40.960517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec3e04-82ce-4c59-b2d6-87d93ea1a009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:40.979413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-11ec3e04-82ce-4c59-b2d6-87d93ea1a009 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:40.980634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11ec3e04-82ce-4c59-b2d6-87d93ea1a009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:40.989611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:08:41.031803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e4619-1773-43ea-892b-4d3c58165574 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:41.050928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e4619-1773-43ea-892b-4d3c58165574 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:41.055465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:41.056111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:41.056390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:41.056668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:41.066708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:41.068337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71114c-dcbb-4854-a55f-99a581c6c96e service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:41.077845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 730/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:43.158676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935fae7-8a9c-42fb-b17a-808fb5af8a7c None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:43.159256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935fae7-8a9c-42fb-b17a-808fb5af8a7c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:43.159256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935fae7-8a9c-42fb-b17a-808fb5af8a7c None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:43.160516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:43.167016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b53be-2925-4ecd-ad57-ac39de1ec1aa None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:43.167789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b53be-2925-4ecd-ad57-ac39de1ec1aa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:43.167953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b53be-2925-4ecd-ad57-ac39de1ec1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:43.199601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-202b53be-2925-4ecd-ad57-ac39de1ec1aa None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:43.201423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-202b53be-2925-4ecd-ad57-ac39de1ec1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:43.215153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:43.264349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935fae7-8a9c-42fb-b17a-808fb5af8a7c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:43.277093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935fae7-8a9c-42fb-b17a-808fb5af8a7c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:43.281712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:43.282363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:43.282658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:43.282938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:43.293217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:43.294761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1523a8-248b-46fe-a0a4-2b50458f4d0f service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:43.304650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 727/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:45.441116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593ff-903d-4ce1-9f64-47796b518df6 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:45.441727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593ff-903d-4ce1-9f64-47796b518df6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:45.441727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50593ff-903d-4ce1-9f64-47796b518df6 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:45.443124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:45.448962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a545ef-d3e8-4e45-b19f-c1fe75e01916 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:45.449746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a545ef-d3e8-4e45-b19f-c1fe75e01916 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:45.449903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a545ef-d3e8-4e45-b19f-c1fe75e01916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:45.472236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c6a545ef-d3e8-4e45-b19f-c1fe75e01916 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:45.473484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c6a545ef-d3e8-4e45-b19f-c1fe75e01916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:45.483416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:45.521429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50593ff-903d-4ce1-9f64-47796b518df6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:45.535652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50593ff-903d-4ce1-9f64-47796b518df6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:45.540371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:45.541015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:45.541293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:45.541566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:45.550782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:45.552404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6d311-eeae-4e7f-b11c-d9dc16eea42f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:45.561737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 733/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:47.665694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e307d-d3ea-47cd-b9e0-25e4c9830156 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:47.666213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e307d-d3ea-47cd-b9e0-25e4c9830156 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:47.666213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e307d-d3ea-47cd-b9e0-25e4c9830156 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:47.667672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:47.673790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc04cd6-1cfe-488e-bee0-60e6563e3ad9 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:47.674257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc04cd6-1cfe-488e-bee0-60e6563e3ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:47.674351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc04cd6-1cfe-488e-bee0-60e6563e3ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:47.682387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=79 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:08:47.696939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3bc04cd6-1cfe-488e-bee0-60e6563e3ad9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:47.698138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3bc04cd6-1cfe-488e-bee0-60e6563e3ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:47.707672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:47.749006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e307d-d3ea-47cd-b9e0-25e4c9830156 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:47.758590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=73 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:08:47.768670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e307d-d3ea-47cd-b9e0-25e4c9830156 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:47.773557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:47.774236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:47.774548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:47.774826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:47.784266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:47.785697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68308888-a1f7-43aa-bda3-5d1abe58b503 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:47.795330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 730/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:49.857348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f797-69f8-4d64-9eca-56cd94278a7e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:49.857936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f797-69f8-4d64-9eca-56cd94278a7e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:49.857936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f797-69f8-4d64-9eca-56cd94278a7e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:49.860141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:49.867460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ea9a-1cf6-47e3-b2e7-01b4b9000917 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:49.867914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ea9a-1cf6-47e3-b2e7-01b4b9000917 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:49.868025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ea9a-1cf6-47e3-b2e7-01b4b9000917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:49.886724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-87d4ea9a-1cf6-47e3-b2e7-01b4b9000917 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:49.887852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-87d4ea9a-1cf6-47e3-b2e7-01b4b9000917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:49.896543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:08:49.944374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f797-69f8-4d64-9eca-56cd94278a7e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:49.959261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f797-69f8-4d64-9eca-56cd94278a7e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:49.963744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:49.964398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:49.964685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:49.964959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:49.974506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:49.975958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64e913-6ab6-4f71-a3ec-d229aa52a9e1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:49.985330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 736/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:08:52.033250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4cdad-6885-4451-afa0-133c588c93ea None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:52.034217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4cdad-6885-4451-afa0-133c588c93ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:52.034217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4cdad-6885-4451-afa0-133c588c93ea None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:52.036335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:08:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:08:52.041808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d90bf-b493-43db-9fcd-f1963298140b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:52.042264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d90bf-b493-43db-9fcd-f1963298140b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:52.042361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d90bf-b493-43db-9fcd-f1963298140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:52.062216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-340d90bf-b493-43db-9fcd-f1963298140b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:08:52.063420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-340d90bf-b493-43db-9fcd-f1963298140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:08:52.072404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:08:52.111158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4cdad-6885-4451-afa0-133c588c93ea None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:08:52.124793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4cdad-6885-4451-afa0-133c588c93ea None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:08:52.129158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:08:52.129802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:08:52.130439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:08:52.130740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:08:52.139948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:08:52.141365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e27ce3-2e82-452b-903f-d313a55ad852 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:08:52.150774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 733/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:02.056430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=11 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:09:02.146535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=14 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:09:11.913174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f821-ee9a-499f-9871-4f66bd09c315 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:11.914001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f821-ee9a-499f-9871-4f66bd09c315 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:11.914001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f821-ee9a-499f-9871-4f66bd09c315 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:11.916268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 738/1471] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:11.921755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55233a0-c01f-4627-8ef4-7a6243872422 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:11.922207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55233a0-c01f-4627-8ef4-7a6243872422 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:11.922311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55233a0-c01f-4627-8ef4-7a6243872422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:11.941869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e55233a0-c01f-4627-8ef4-7a6243872422 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:11.943077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e55233a0-c01f-4627-8ef4-7a6243872422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:11.952208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 734/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:12.046793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aeda3-cf5a-4eff-b52c-dbd66e1fe352 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:12.047081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aeda3-cf5a-4eff-b52c-dbd66e1fe352 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:12.047354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aeda3-cf5a-4eff-b52c-dbd66e1fe352 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:12.048774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 739/1473] 10.222.0.16 () {54 vars in 962 bytes} [Mon Jul 4 04:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:12.054679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde604e-aeb2-4581-9347-1be122052353 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:12.055067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde604e-aeb2-4581-9347-1be122052353 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:12.055463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde604e-aeb2-4581-9347-1be122052353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:12.079609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fbde604e-aeb2-4581-9347-1be122052353 None None] MFA Rules not processed for user `dcab788fe9b0499c99336de1c06c885d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:12.080801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbde604e-aeb2-4581-9347-1be122052353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:12.088808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 735/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Mon Jul 4 04:09:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:12.096311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aeda3-cf5a-4eff-b52c-dbd66e1fe352 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:12.109626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aeda3-cf5a-4eff-b52c-dbd66e1fe352 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:12.114099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:12.114767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:12.115045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:12.115324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:12.125283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:12.126816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264539a2-e932-4de5-b85c-403552ed68cf service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:12.136935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 740/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:14.262145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627e4e-0c65-4ddb-a481-86ddd30aad27 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:14.263437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627e4e-0c65-4ddb-a481-86ddd30aad27 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:14.263618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627e4e-0c65-4ddb-a481-86ddd30aad27 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:14.265822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 736/1476] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:14.271736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f59fa5-fccd-48a9-a2d5-490a8b00faa1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:14.272170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f59fa5-fccd-48a9-a2d5-490a8b00faa1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:14.272290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f59fa5-fccd-48a9-a2d5-490a8b00faa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:14.292158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-65f59fa5-fccd-48a9-a2d5-490a8b00faa1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:14.293343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-65f59fa5-fccd-48a9-a2d5-490a8b00faa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:14.302810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 741/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:09:14.334465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00627e4e-0c65-4ddb-a481-86ddd30aad27 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:14.350654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00627e4e-0c65-4ddb-a481-86ddd30aad27 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:14.356099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:14.356747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:14.357029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:14.357299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:14.358043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:14.359869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:09:14.359869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:09:14.359869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:09:14.360999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bc31a-b537-4fb2-a834-601c8a5036f9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:14.380973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 737/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Mon Jul 4 04:09:14 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:09:16.413402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d4d6-07f8-4e01-b459-1d42ac79cf8c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:16.414280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d4d6-07f8-4e01-b459-1d42ac79cf8c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:16.414280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d4d6-07f8-4e01-b459-1d42ac79cf8c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:16.416693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 742/1479] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:16.422391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272ca6-915f-4934-ad30-40ee51930eb3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:16.422921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272ca6-915f-4934-ad30-40ee51930eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:16.422921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272ca6-915f-4934-ad30-40ee51930eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:16.443676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-48272ca6-915f-4934-ad30-40ee51930eb3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:16.445019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-48272ca6-915f-4934-ad30-40ee51930eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:16.453992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 738/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:16.498500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9d4d6-07f8-4e01-b459-1d42ac79cf8c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:16.517505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9d4d6-07f8-4e01-b459-1d42ac79cf8c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:16.523416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:16.524060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:16.524341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:16.524614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:16.532769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:16.534352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd73ed3-6d85-4a2a-a9af-58825eaf8686 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:16.543785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 743/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:18.762544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e783c-2173-40a7-9f22-3c89297b699d None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:18.763152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e783c-2173-40a7-9f22-3c89297b699d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:18.763152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e783c-2173-40a7-9f22-3c89297b699d None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:18.765234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 739/1482] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:18.770713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7f2c-8679-43b5-8aef-2c3501762aa5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:18.771160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7f2c-8679-43b5-8aef-2c3501762aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:18.771258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7f2c-8679-43b5-8aef-2c3501762aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:18.790868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ccbf7f2c-8679-43b5-8aef-2c3501762aa5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:18.792132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccbf7f2c-8679-43b5-8aef-2c3501762aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:18.800439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 744/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:18.837056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e783c-2173-40a7-9f22-3c89297b699d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:18.852188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e783c-2173-40a7-9f22-3c89297b699d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:18.856357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:18.856998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:18.857274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:18.857543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:18.867069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:18.868499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e5db-3c8b-47a9-99ce-a028c0f7e69a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:18.879762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 740/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:20.949676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0a7e-d20a-4682-b29e-e8d570e78036 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:20.950151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0a7e-d20a-4682-b29e-e8d570e78036 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:20.950441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0a7e-d20a-4682-b29e-e8d570e78036 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:20.952122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 745/1485] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:20.958184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845dd7-b763-4ccb-8f11-7c4996a66d38 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:20.959029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845dd7-b763-4ccb-8f11-7c4996a66d38 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:20.959312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845dd7-b763-4ccb-8f11-7c4996a66d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:20.985097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-86845dd7-b763-4ccb-8f11-7c4996a66d38 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:20.986303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-86845dd7-b763-4ccb-8f11-7c4996a66d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:20.995389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 741/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:21.040678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e0a7e-d20a-4682-b29e-e8d570e78036 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:21.053235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e0a7e-d20a-4682-b29e-e8d570e78036 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:21.057455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:21.058030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:21.058340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:21.058643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:21.068430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:21.069858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f354d52-d335-416d-9e02-2bf6c0a4806c service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:21.080114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 746/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:23.191487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4e20-4d98-4f4b-af38-0b68b42573fa None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:23.192274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4e20-4d98-4f4b-af38-0b68b42573fa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:23.192274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4e20-4d98-4f4b-af38-0b68b42573fa None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:23.194446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 742/1488] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:23.201358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02949d39-8ab8-453b-a203-695e27a39c49 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:23.201834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02949d39-8ab8-453b-a203-695e27a39c49 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:23.201933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02949d39-8ab8-453b-a203-695e27a39c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:23.220132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-02949d39-8ab8-453b-a203-695e27a39c49 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:23.221331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-02949d39-8ab8-453b-a203-695e27a39c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:23.230290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 747/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:09:23.273584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4e20-4d98-4f4b-af38-0b68b42573fa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:23.288551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4e20-4d98-4f4b-af38-0b68b42573fa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:23.294759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:23.295414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:23.295717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:23.295993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:23.306399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:23.308422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45bfc4-22b1-4ea0-8cc5-1fa4761bb37c service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:23.318401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 743/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:09:25.567651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd0fcb-69c3-494a-bfb9-7e57ff033993 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:25.568263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd0fcb-69c3-494a-bfb9-7e57ff033993 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:25.568263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd0fcb-69c3-494a-bfb9-7e57ff033993 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:25.569568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 748/1491] 10.222.0.16 () {54 vars in 926 bytes} [Mon Jul 4 04:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:09:25.576080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ebe6-2823-4f37-a735-3a5bd012d6bf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:25.576537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ebe6-2823-4f37-a735-3a5bd012d6bf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:25.576661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ebe6-2823-4f37-a735-3a5bd012d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:25.606011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4322ebe6-2823-4f37-a735-3a5bd012d6bf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:09:25.607263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4322ebe6-2823-4f37-a735-3a5bd012d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:09:25.615284 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 744/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Mon Jul 4 04:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:09:25.653837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd0fcb-69c3-494a-bfb9-7e57ff033993 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:09:25.664768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd0fcb-69c3-494a-bfb9-7e57ff033993 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:09:25.668777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:09:25.669421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:09:25.669696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:09:25.669972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:09:25.679541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:09:25.680972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d222c1-90cd-4577-aa41-90d684e9742f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:09:25.691001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 749/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:09:35.596874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=33 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:09:35.686363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=33 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:10:51.890497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:51.891087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:51.891087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:51.910892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:10:51.912086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:51.921205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 745/1494] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:10:51.929972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:51.930517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:51.930517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:51.950187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:10:51.951417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:51.960837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 750/1495] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:10:51.965446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:51.975085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac45975-78f5-4364-a8ea-5fd45a574aa8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:51.980131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:51.980950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:51.981238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:51.981516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:51.982230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:10:51.982564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:51.983586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:51.983586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:10:51.983586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:10:51.984464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:51.996169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 746/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:10:52.004604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.018813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694d7c2-d990-4fed-87c4-0dace0744391 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.027469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.028166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.028445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.028715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.029421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.031250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.031250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:10:52.031250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:10:52.032495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.072435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 751/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:10:52 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:10:52.078243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.092514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f8e4e-82b8-4e5b-a48c-232b34807269 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.149763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.150348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.150660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.150937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.151629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.152713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.152713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:10:52.152713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:10:52.153899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.176482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 747/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:10:52 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:10:52.181234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.192445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ae18-23b5-44c3-a720-121b1de12c35 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.197205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.197860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.198162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.198442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.199182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.200203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.200203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:10:52.200203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:10:52.201068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.214096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 752/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:10:52.218043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.228068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a28aaf-5ec4-42c0-b79e-d60105b549f5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.233024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.233682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.233966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.234248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.235014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.236071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.236071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:10:52.236071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:10:52.236943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaea86-f12f-4534-86d8-eea7b935cbb7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.250349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 748/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:10:52.254658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.265334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040749d-bca3-4c81-bc25-773ebf835897 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.271117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.272010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.272290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.272566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] PATH_INFO: `/v3/projects/fe64258a07fd4974aeb91d21b4a70f31/users/83ef513f1d00433bb9d6670bdd108c5d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.288194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe64258a07fd4974aeb91d21b4a70f31, user_id=83ef513f1d00433bb9d6670bdd108c5d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.289228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.289228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:10:52.289228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:10:52.291167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5163f-0288-4a54-9801-868d32f4f970 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.306916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 753/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:10:52 2022] PUT /identity/v3/projects/fe64258a07fd4974aeb91d21b4a70f31/users/83ef513f1d00433bb9d6670bdd108c5d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:10:52.326035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea2b2f-7b38-4922-ba0a-24808e09c8ac None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.326323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea2b2f-7b38-4922-ba0a-24808e09c8ac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.326645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea2b2f-7b38-4922-ba0a-24808e09c8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.354400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-87ea2b2f-7b38-4922-ba0a-24808e09c8ac None None] MFA Rules not processed for user `83ef513f1d00433bb9d6670bdd108c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:10:52.355801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-87ea2b2f-7b38-4922-ba0a-24808e09c8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:52.428417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 749/1502] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:10:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:10:52.442738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac408-30da-4fac-ab7a-a178ab557178 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.443024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac408-30da-4fac-ab7a-a178ab557178 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.443300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac408-30da-4fac-ab7a-a178ab557178 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.444655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 754/1503] 10.222.0.16 () {54 vars in 963 bytes} [Mon Jul 4 04:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:10:52.449856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041bce-3c83-4496-933c-753daab89b03 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.450139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041bce-3c83-4496-933c-753daab89b03 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.450454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041bce-3c83-4496-933c-753daab89b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.468189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-08041bce-3c83-4496-933c-753daab89b03 None None] MFA Rules not processed for user `0f9c5095b2ac44c1833141d0949c6ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:10:52.469366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-08041bce-3c83-4496-933c-753daab89b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:52.524215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:10:52.534717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 750/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Mon Jul 4 04:10:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:10:52.542622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bac408-30da-4fac-ab7a-a178ab557178 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.555072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bac408-30da-4fac-ab7a-a178ab557178 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.559005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.559637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.559910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.560191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.568539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.569963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.582152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 755/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:10:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:10:52.625179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7eecd-3263-48e4-ab16-dd4ac42264d9 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.625471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7eecd-3263-48e4-ab16-dd4ac42264d9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.625750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7eecd-3263-48e4-ab16-dd4ac42264d9 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.627301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 751/1506] 10.222.0.16 () {52 vars in 829 bytes} [Mon Jul 4 04:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:10:52.631047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.696625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c597a23-99cb-4d18-93cd-77dcca95e30f service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.702192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.703092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.703376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.703655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] PATH_INFO: `/v3/users/83ef513f1d00433bb9d6670bdd108c5d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.704905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ef513f1d00433bb9d6670bdd108c5d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:52.705936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:52.705936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:10:52.705936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:10:52.707165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:52.776121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ef513f1d00433bb9d6670bdd108c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:52.789212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ef513f1d00433bb9d6670bdd108c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:52.789939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ef513f1d00433bb9d6670bdd108c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:52.802846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ef513f1d00433bb9d6670bdd108c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:52.819716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:10:52.822684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ef513f1d00433bb9d6670bdd108c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:52.830424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 756/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:10:52 2022] DELETE /identity/v3/users/83ef513f1d00433bb9d6670bdd108c5d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:10:52.834897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.835187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.835466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:52.852757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:10:52.853954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:52.916131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 752/1508] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:10:52.927068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:52.928758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:52.987125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad80c-9c87-4013-90ff-b1933bb13953 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:52.991299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:52.992006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:52.992323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:52.992604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:53.002801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:53.004314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4c76-8f92-41fe-9da8-7e437f1cec66 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:53.014606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 757/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:10:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:10:53.408078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:10:53.409728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:10:53.422599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab7bb9-dc4f-4ec0-b33e-e01c3aae86bb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:10:53.428575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:10:53.430015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:10:53.430307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:10:53.430722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] PATH_INFO: `/v3/projects/fe64258a07fd4974aeb91d21b4a70f31` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:10:53.431810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe64258a07fd4974aeb91d21b4a70f31)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:10:53.433852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:10:53.433852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:10:53.433852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:10:53.435474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:10:53.501024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe64258a07fd4974aeb91d21b4a70f31', 'request_id': 'req-6950d168-5bc0-486b-a893-56c5bd3079ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:53.516340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe64258a07fd4974aeb91d21b4a70f31', 'request_id': 'req-6950d168-5bc0-486b-a893-56c5bd3079ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:53.522968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe64258a07fd4974aeb91d21b4a70f31', 'request_id': 'req-6950d168-5bc0-486b-a893-56c5bd3079ed'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:53.537547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] The token cache is being invalidate because project fe64258a07fd4974aeb91d21b4a70f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:10:53.537832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:53.538539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6950d168-5bc0-486b-a893-56c5bd3079ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:10:53.540363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 753/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:10:53 2022] DELETE /identity/v3/projects/fe64258a07fd4974aeb91d21b4a70f31 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:11:03.009734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:11:03.534439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:12:31.576852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:31.577495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:31.577495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:31.578825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:31.584208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec45c-0ebe-4e5f-83bb-dbfbbca8d09f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:31.584670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec45c-0ebe-4e5f-83bb-dbfbbca8d09f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:31.584870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec45c-0ebe-4e5f-83bb-dbfbbca8d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:31.603719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-eaaec45c-0ebe-4e5f-83bb-dbfbbca8d09f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:31.604933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eaaec45c-0ebe-4e5f-83bb-dbfbbca8d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:31.685070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:31.852501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:31.854554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:31.938767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60780d1-b72a-428a-9667-ec2034e761f7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:31.945024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:31.945688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:31.945969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:31.946250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:31.956152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:31.957732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c4ac1-31f2-402a-aadd-b4a965c40eb6 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:31.967761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 759/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:32.833929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bd9a-8c17-4521-a490-5fa5576d8df2 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:32.834560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bd9a-8c17-4521-a490-5fa5576d8df2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:32.834560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bd9a-8c17-4521-a490-5fa5576d8df2 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:32.836029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:32.842268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d992878-c616-4127-b5b4-d85c93d37515 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:32.843046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d992878-c616-4127-b5b4-d85c93d37515 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:32.843217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d992878-c616-4127-b5b4-d85c93d37515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:32.868200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5d992878-c616-4127-b5b4-d85c93d37515 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:32.869344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5d992878-c616-4127-b5b4-d85c93d37515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:32.878994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:33.031347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bd9a-8c17-4521-a490-5fa5576d8df2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:33.047244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bd9a-8c17-4521-a490-5fa5576d8df2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:33.053165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:33.053805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:33.054091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:33.054387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:33.063515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:33.065127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c6a62-e79d-4399-8334-931a7caddd6a service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:33.075453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 756/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:33.527616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1237db-202a-4e2c-892a-dc8203fe8771 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:33.527909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1237db-202a-4e2c-892a-dc8203fe8771 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:33.528187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1237db-202a-4e2c-892a-dc8203fe8771 None None] PATH_INFO: `/v3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:33.529515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 761/1517] 10.222.0.59 () {54 vars in 925 bytes} [Mon Jul 4 04:12:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:12:33.537271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716051c3-fa5d-4408-97a5-c8b6abd5b221 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:33.537528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716051c3-fa5d-4408-97a5-c8b6abd5b221 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:33.537916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716051c3-fa5d-4408-97a5-c8b6abd5b221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:33.560952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-716051c3-fa5d-4408-97a5-c8b6abd5b221 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:33.562134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-716051c3-fa5d-4408-97a5-c8b6abd5b221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:33.633992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 757/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Jul 4 04:12:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:42.560966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfab98-f518-437a-ba95-3310498d6b1e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:42.562041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfab98-f518-437a-ba95-3310498d6b1e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:42.562041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfab98-f518-437a-ba95-3310498d6b1e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:42.562758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 762/1519] 10.222.0.16 () {54 vars in 960 bytes} [Mon Jul 4 04:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:42.568592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7933-2670-4a65-830d-d5de71b12a09 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:42.569046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7933-2670-4a65-830d-d5de71b12a09 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:42.569144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7933-2670-4a65-830d-d5de71b12a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:42.591466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-13cd7933-2670-4a65-830d-d5de71b12a09 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:42.593359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-13cd7933-2670-4a65-830d-d5de71b12a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:42.606036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 758/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Mon Jul 4 04:12:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:42.614558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfab98-f518-437a-ba95-3310498d6b1e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:42.629359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfab98-f518-437a-ba95-3310498d6b1e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:42.634552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:42.635251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:42.635539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:42.635820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:42.645591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:42.647407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2234af-081e-4ecc-adee-f93e0a1cdd63 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:42.657241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 763/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:44.326323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d93a1-3bf2-40ca-8c6c-9815cfa9e4af None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:44.327349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d93a1-3bf2-40ca-8c6c-9815cfa9e4af None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:44.327349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d93a1-3bf2-40ca-8c6c-9815cfa9e4af None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:44.329819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:12:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:44.338117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcbdd-061a-4e91-b45f-942dac542efc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:44.338939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcbdd-061a-4e91-b45f-942dac542efc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:44.339244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcbdd-061a-4e91-b45f-942dac542efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:44.361334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-dcfdcbdd-061a-4e91-b45f-942dac542efc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:44.362584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dcfdcbdd-061a-4e91-b45f-942dac542efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:44.370981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:44.528880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d93a1-3bf2-40ca-8c6c-9815cfa9e4af None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:44.550599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d93a1-3bf2-40ca-8c6c-9815cfa9e4af None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:44.557126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:44.557801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:44.558055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:44.558328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:44.566959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:44.568556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1784a-11d9-4f7a-a861-4e9404cb7101 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:44.577881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 760/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:45.403022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c229f8-d9ef-4808-96b7-95a30cbd4561 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:45.403022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c229f8-d9ef-4808-96b7-95a30cbd4561 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:45.403736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c229f8-d9ef-4808-96b7-95a30cbd4561 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:45.404715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 04:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:45.410884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017bbc0-f48b-40fa-a929-da67af6d0f25 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:45.411342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017bbc0-f48b-40fa-a929-da67af6d0f25 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:45.411463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017bbc0-f48b-40fa-a929-da67af6d0f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:45.438693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b017bbc0-f48b-40fa-a929-da67af6d0f25 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:45.439903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b017bbc0-f48b-40fa-a929-da67af6d0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:45.451946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 04:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:45.607518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c229f8-d9ef-4808-96b7-95a30cbd4561 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:45.633936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c229f8-d9ef-4808-96b7-95a30cbd4561 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:45.642196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:45.642893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:45.643180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:45.643444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:45.652661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:45.654873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87b65f-9928-488b-8c2f-8f2c97895932 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:45.664170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 766/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:49.514350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:49.515308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:49.515532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:49.518575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:12:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:49.524177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f078-271c-46d5-aee7-910fdb7e1e37 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:49.524622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f078-271c-46d5-aee7-910fdb7e1e37 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:49.524807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f078-271c-46d5-aee7-910fdb7e1e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:49.553710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1606f078-271c-46d5-aee7-910fdb7e1e37 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:49.555456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1606f078-271c-46d5-aee7-910fdb7e1e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:49.569876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:49.651940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:49.655162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:49.715621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ea67a-a41c-4d07-9fd5-0a354822ba44 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:49.719724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:49.720417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:49.720693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:49.720959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:49.730654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:49.732142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbf394-52f0-4ca7-87e6-44bc5db23ce0 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:49.737357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:12:49.743079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 763/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:51.774569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:51.775204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:51.775204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:51.776540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:51.782137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb61-1547-45c7-8da1-3a1d39bfd6a0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:51.782650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb61-1547-45c7-8da1-3a1d39bfd6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:51.782767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb61-1547-45c7-8da1-3a1d39bfd6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:51.803321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6cc7cb61-1547-45c7-8da1-3a1d39bfd6a0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:51.804563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6cc7cb61-1547-45c7-8da1-3a1d39bfd6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:51.812995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:51.855450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:51.858100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:51.874218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a18b3e-67ab-4b46-be8a-85f31041da69 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:51.878643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:51.879294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:51.879563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:51.879840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:51.890583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:51.891967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672441-7894-4ad9-a905-f7bbd99d7fad service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:51.901652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 769/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:55.340799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:55.341443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:55.341443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:55.343441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:55.351213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240593fd-3324-40be-a630-5b7ac1d70842 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:55.351725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240593fd-3324-40be-a630-5b7ac1d70842 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:55.351725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240593fd-3324-40be-a630-5b7ac1d70842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:55.359520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=62 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:12:55.373516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-240593fd-3324-40be-a630-5b7ac1d70842 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:55.374750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-240593fd-3324-40be-a630-5b7ac1d70842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:55.384473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:55.705597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:55.708666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:55.775643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb369c61-e821-4a7f-ba7c-dcd0c2c94fcc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:55.779530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:55.780189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:55.780471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:55.780749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:55.795645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:55.799411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa5b4b-90c7-4be7-81d4-2660011c3eb6 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:55.815936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 766/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:12:57.875872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:57.876648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:57.876648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:57.877838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 04:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:12:57.884571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f8d15-350b-4d6f-99b2-1b9411cd63b3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:57.885302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f8d15-350b-4d6f-99b2-1b9411cd63b3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:57.885472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f8d15-350b-4d6f-99b2-1b9411cd63b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:57.913757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d64f8d15-350b-4d6f-99b2-1b9411cd63b3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:12:57.915010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d64f8d15-350b-4d6f-99b2-1b9411cd63b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:57.924179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 04:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:12:58.230321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:12:58.232416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:12:58.253950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d268f-93a1-4d61-9380-0c36230386fb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:12:58.261533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:12:58.262643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:12:58.263379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:12:58.263828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:12:58.274398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:12:58.275847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bc242-fe18-4522-a403-c07013b27f6c service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:12:58.285922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 772/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:13:07.908043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=32 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:13:08.281454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=8 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:26.319487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b531-8e44-47bd-a628-393b460c9617 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.320171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b531-8e44-47bd-a628-393b460c9617 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.320171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b531-8e44-47bd-a628-393b460c9617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.343363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0a46b531-8e44-47bd-a628-393b460c9617 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.344689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0a46b531-8e44-47bd-a628-393b460c9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.357250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 768/1540] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:14:26.370758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.371369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.371369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.395413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.396652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.405227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 773/1541] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:14:26.409933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e81f-0d39-499d-a76b-fbf6400fb44e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.410144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e81f-0d39-499d-a76b-fbf6400fb44e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.410336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e81f-0d39-499d-a76b-fbf6400fb44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.417142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.446394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ef94d-18f9-4cd9-b9f8-1556c602649f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.448798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d209e81f-0d39-499d-a76b-fbf6400fb44e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.450014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d209e81f-0d39-499d-a76b-fbf6400fb44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.459273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.459980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.460265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.460545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.461316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:26.461632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.462688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:26.462688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:14:26.462688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:14:26.463587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.469558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 769/1542] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.480012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbfae-16a3-437b-8d45-8b8997ac0e89 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.480379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbfae-16a3-437b-8d45-8b8997ac0e89 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.480576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbfae-16a3-437b-8d45-8b8997ac0e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.488540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 774/1543] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:26.498019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.511780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bcdbbfae-16a3-437b-8d45-8b8997ac0e89 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.513008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bcdbbfae-16a3-437b-8d45-8b8997ac0e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.515876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f386-3345-497b-b1de-72f4d7de9eab admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.527468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 770/1544] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.529016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.529253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.529502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.529705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.529823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.529994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.530258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.531349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.533298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0427e-dc3e-4e71-8eb3-af7c2297a0b0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.557057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.558260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.567962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 771/1545] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.568908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.579853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9ee82-650c-4c86-aace-4b7aa9045433 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.584612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.584962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 775/1546] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:26.585448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.585735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.586013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.586354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b6d3a-54a4-478a-a8c2-beca36a2e0a8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.586682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b6d3a-54a4-478a-a8c2-beca36a2e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.586937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:26.587049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b6d3a-54a4-478a-a8c2-beca36a2e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.587242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.589124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800c680-4b88-4d75-9b46-c7e85bb8c303 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.605273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 772/1547] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:26.606905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.607200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.607480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.608472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-277b6d3a-54a4-478a-a8c2-beca36a2e0a8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.610063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-277b6d3a-54a4-478a-a8c2-beca36a2e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.638537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.640572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.674478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 776/1548] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.676171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036d9c-149d-4544-9ddf-5e2aa4c149d5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.676453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036d9c-149d-4544-9ddf-5e2aa4c149d5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.676727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036d9c-149d-4544-9ddf-5e2aa4c149d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.693860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0f036d9c-149d-4544-9ddf-5e2aa4c149d5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.695100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f036d9c-149d-4544-9ddf-5e2aa4c149d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.702702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 777/1549] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.703489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 773/1550] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.703902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a90b-2a8b-4ca8-af12-f5106ea6b438 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.704180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.704311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a90b-2a8b-4ca8-af12-f5106ea6b438 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.704482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a90b-2a8b-4ca8-af12-f5106ea6b438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.719049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc963f-cb63-4b4c-9734-20f7835104ea None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.723999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.724673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.724938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.725208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.725911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.727853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.728416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0724a90b-2a8b-4ca8-af12-f5106ea6b438 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.729599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0724a90b-2a8b-4ca8-af12-f5106ea6b438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.737431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 778/1551] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.738817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9675-f036-467d-b0ac-f8b53e797b31 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.739105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9675-f036-467d-b0ac-f8b53e797b31 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.739377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9675-f036-467d-b0ac-f8b53e797b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.756286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 774/1552] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/users => generated 297 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:26.757233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.759308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f05a9675-f036-467d-b0ac-f8b53e797b31 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.760665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f05a9675-f036-467d-b0ac-f8b53e797b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.767072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40133e7e-c38f-4d0f-b2c0-2b57dde2628c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.769118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 779/1553] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.770382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89de621-ea89-49e4-88c7-81812337c867 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.770678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89de621-ea89-49e4-88c7-81812337c867 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.770959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89de621-ea89-49e4-88c7-81812337c867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.778161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.778903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.779268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.779526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.780226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.781253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:26.781253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:14:26.781253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:14:26.782525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.788555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d89de621-ea89-49e4-88c7-81812337c867 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.789772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d89de621-ea89-49e4-88c7-81812337c867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.798917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 780/1554] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.800362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be81538-5d9b-471f-bd8b-bb037b3a25c4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.800675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be81538-5d9b-471f-bd8b-bb037b3a25c4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.801050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be81538-5d9b-471f-bd8b-bb037b3a25c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.814982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 775/1555] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:26.814982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.821315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3be81538-5d9b-471f-bd8b-bb037b3a25c4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.822556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3be81538-5d9b-471f-bd8b-bb037b3a25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.876639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181718-ff24-4992-be72-03964f599728 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.882911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 781/1556] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.884039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.885491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.886408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.887111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.887242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.887242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.887437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.887996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:26.888342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.890440 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43218c3f-1d52-412b-98a2-a8fe826ac02e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.903144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 776/1557] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:26.904446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc60b3-e351-4c49-b854-ecfe955a49e4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.904735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc60b3-e351-4c49-b854-ecfe955a49e4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.905009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc60b3-e351-4c49-b854-ecfe955a49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.914992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.916172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.922738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fbcc60b3-e351-4c49-b854-ecfe955a49e4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.923907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbcc60b3-e351-4c49-b854-ecfe955a49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.924563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 782/1558] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.925309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.934814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1980eb1-f265-458d-842e-b0d11ccaf672 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.935836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 777/1559] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.936971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.937258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.937568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.943488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.944167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.944457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.944731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.945731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.947198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfb35a-c75a-4785-bb72-465b734863e6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.955992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.957399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 783/1560] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:26.957523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.958669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.958955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.959246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.966864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 778/1561] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.967657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.972412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:26.977558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:26.978478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b885-2679-4af1-92fb-9edb39e3aab2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:26.978795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:26.984722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:26.985646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:26.985930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:26.986244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:26.986949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:26.987387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 784/1562] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:26.988313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:26.989443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:26.998090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912744cd-8263-4d88-81e1-df867c37b80a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.005083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.005740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.006021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.006307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.007349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:27.007660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.009283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.011801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:27.012625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 779/1563] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:26 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.013536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.023923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686f282-9db6-4e7c-997e-45173955f9c3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.031789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 785/1564] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:27.031789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.034199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.034955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.035240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.035528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.036346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.038294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.041575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99addd-8b9f-4b0f-9ce8-8f5ca3d647ca admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.046419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.047082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.047363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.047642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.048438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:27.048741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.050482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.063699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 786/1565] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:27.064761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.067068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 780/1566] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.067754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.078616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf283fd-f0dc-478c-8a59-2c1d2ae5baa4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.128784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c2ddf-5e0e-497c-b1b6-2420ef710581 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.133181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.133590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.133843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.134126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.134258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.134405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.134570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.134844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.135144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.135570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:27.135876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.136756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.137438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.147404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 781/1567] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.148194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.155788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 787/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:27.155788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.161458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f626c-17fc-479d-99b4-3e0522e4cf85 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.162220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd3ba-686c-4ae9-a0e1-10122046a7b5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.166197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.166880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.167017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.167155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.167428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.167591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.167878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.168150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.168271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:27.168452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.169026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:27.169328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.170066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.170941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.182352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 782/1569] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:27.183160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.191890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 788/1570] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:27.191890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.194585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afd36f-5040-4585-bde8-f879875ef195 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.197496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b37af0-29e0-447f-9c37-90bcd7840540 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.199131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.199794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.200102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.200375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.201113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.202719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.202857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.203377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.203659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.203933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.204628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.206585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.213344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 783/1571] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.214178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.223792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251f0c-ed54-4571-ba7d-ba8bec50a5f0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.229267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.229919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.230199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.230505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.231196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.233111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.236539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 789/1572] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.237983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.255227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 784/1573] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.256114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.267302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962d0e2-f305-4e19-a7ac-e64f033769eb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.304528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106531-fded-4f0d-9a0b-4d8d4a50af79 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.309833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.310570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.310857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.311138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.311840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.313775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.324804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.325711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.325987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.326259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] PATH_INFO: `/v3/projects/7ea328a1154a4a9a955e411f7eab0aa9/users/dca54bc220104e79aa727f30399f3411/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.342115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 790/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.342959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.347432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea328a1154a4a9a955e411f7eab0aa9, user_id=dca54bc220104e79aa727f30399f3411, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.348890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:27.348890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:14:27.348890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:14:27.350877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.360536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:27.371681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 785/1575] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:27 2022] PUT /identity/v3/projects/7ea328a1154a4a9a955e411f7eab0aa9/users/dca54bc220104e79aa727f30399f3411/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:27.371681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.377651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:27.416099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49895446-340b-4833-b4b3-dd546296a45c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.431142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945709a3-b5cc-41a7-b480-48f0d47b6693 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.436163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.436913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.437225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.437464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.438202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.440137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.466857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 786/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.467496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.476872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.477546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.477829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.478109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.478876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.480781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.509335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 791/1577] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.510775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.529306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101313-a7b0-4d0d-8d9a-a0cd944e5424 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.571539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf193d-8e99-488d-b022-c45fb2d30fe8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.576578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.577250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.577527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.577798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.578562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.580240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.583959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.584633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.584918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.585192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.585884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.587792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.594187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 792/1578] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.594187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.601903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a32d0a-13ea-444e-91fa-a6ccf9339b1c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.606405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.607196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.607542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.607899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.608660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.609914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:27.609914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:14:27.609914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:14:27.611205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.620744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 787/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.621476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.640703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 793/1580] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.641578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.646170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:27.652941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92acb0-916b-4253-8bfc-4eadbab045a3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.688052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:27.692039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421107-0064-467e-8f72-97854a643a1d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.696756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.697469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.697747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.698028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.698817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.700482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f4d48-1b38-4fc3-b1a7-3353b3c7c04c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.711259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 788/1581] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.712491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378e87-4336-4dcc-a421-88413d248847 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.712780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378e87-4336-4dcc-a421-88413d248847 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.713061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378e87-4336-4dcc-a421-88413d248847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.716329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.717022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.717308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.717610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.718311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.720262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.730961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-60378e87-4336-4dcc-a421-88413d248847 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:27.732418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-60378e87-4336-4dcc-a421-88413d248847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:27.741399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 789/1582] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:27.742301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60378e87-4336-4dcc-a421-88413d248847 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.746465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 794/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.746465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.752641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60378e87-4336-4dcc-a421-88413d248847 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.756008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376b75-3a95-43f9-9206-c46effeb014f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.757313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.757989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.758273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.758576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.759267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.761174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.761281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.761876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.762160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.762486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.763173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.765382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.783091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 790/1584] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.786445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 795/1585] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.787164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.787284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.797076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30f8e0-538b-405d-9ef7-c15296613d6c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.797225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c4279-1eeb-4dcb-8c54-e2e01a8f27ce admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.801748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.802011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.802418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.802710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.802894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.803041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.803178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.803483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] PATH_INFO: `/v3/projects/7ccd298eebe6413daaae0d801e902616/users/4041faa2919d4760ae10907070a10dc7/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.803622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.805501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.819891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ccd298eebe6413daaae0d801e902616, user_id=4041faa2919d4760ae10907070a10dc7, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.823039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.827335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 796/1586] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:27 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:27.828130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.838031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39daf8c-fae9-446e-9484-efd1d86523ae admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.840786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 791/1587] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:27 2022] PUT /identity/v3/projects/7ccd298eebe6413daaae0d801e902616/users/4041faa2919d4760ae10907070a10dc7/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:27.841685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.897272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.897943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.898283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.898587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.899318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.901044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.904814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807334c-9e98-426a-b24c-d3c9eca04ee3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.909401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.910276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.910606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.910880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.911612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.912511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 797/1588] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.913200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.913426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.924235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b57f45-9be7-484f-94cd-456a8f27488e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.924520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 792/1589] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.925453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.929279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.929952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.930231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.930534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.931521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.933086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.946518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 798/1590] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.946518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.954143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818b89-f349-4cc2-94d6-0acbc1df0a29 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.959151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.959783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.960057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.960339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.961035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.962640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.973793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 799/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:27.974755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:27.984964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7941cc-7dfa-4c38-81b4-f0c2c9d6dfe1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.989667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76079722-886e-48d1-aa0d-2f272c0c6c9b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:27.990257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.990964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.991246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.991522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:27.992274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:27.993377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763050-c034-4924-a219-28480524730c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:27.993862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:27.994034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763050-c034-4924-a219-28480524730c service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:27.994319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763050-c034-4924-a219-28480524730c service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:27.994630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763050-c034-4924-a219-28480524730c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.003763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763050-c034-4924-a219-28480524730c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.004982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 800/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.005296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763050-c034-4924-a219-28480524730c service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.005894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.015282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 793/1593] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:28.015959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4388-4b9e-4746-b4ca-b36b9a7301cb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.016767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.017180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.017467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.022552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.024123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.024483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.024841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.025713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.026735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:28.027566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.040474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 801/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.041451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.044421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:28.045687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:28.054836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 794/1595] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:28.055017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd21d-8c3e-4150-90f3-667def5d429c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.058619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.059763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.060413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.060691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.060969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.061983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.063912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.069266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173ad1-8af5-4b77-8d8b-956762035e2c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.075029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.075741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 802/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.076031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.076369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.076657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] PATH_INFO: `/v3/projects/16c2e6487a1b435c9a362c1fd52b9978/users/c756a1bc870a4ec5a5b629c91e6e84a3/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.079682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.090501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad74c-73e3-4131-8c58-cfb5ef28d9a0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.095000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c2e6487a1b435c9a362c1fd52b9978, user_id=c756a1bc870a4ec5a5b629c91e6e84a3, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.095614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.096293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.096581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.096866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.097973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.098912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.099791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.111459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 803/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.112138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.131390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 795/1598] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/16c2e6487a1b435c9a362c1fd52b9978/users/c756a1bc870a4ec5a5b629c91e6e84a3/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.131390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.136844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7b73f-866e-4727-9ad6-da48cbc16bc4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.163303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:28.205884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ed9-abe5-44ce-8f27-6eb73d79d551 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.210711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.211403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.211693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.212003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.213045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.214429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.214713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.215052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.215392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.215831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.216988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.219038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.231013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 796/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.231434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.241466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 804/1600] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.242560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.248411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbe3b1-593c-42a5-a270-30611add7f39 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.254307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.255114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.255506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.255847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.256688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.258427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.273379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 797/1601] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:28.275079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.286790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e617-793f-456c-8b73-855c3098d4b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.291662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.292647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.292992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.293348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] PATH_INFO: `/v3/projects/94810165aeb641848b753c96ca3ad7a8/users/1758d6fed42543fc9ff569b7051cb3e8/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.308407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6fa95-0de5-4ecc-b0f4-105bec1e6569 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.312378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.312897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94810165aeb641848b753c96ca3ad7a8, user_id=1758d6fed42543fc9ff569b7051cb3e8, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.313076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.313327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.313606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.316259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.329849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.330470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a30d1-49ae-4e40-b96e-40c0ff140c2a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.343714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 798/1602] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/94810165aeb641848b753c96ca3ad7a8/users/1758d6fed42543fc9ff569b7051cb3e8/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.343714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.354461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 805/1603] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:28.356269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.356904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.357655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.395393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] MFA Rules not processed for user `c756a1bc870a4ec5a5b629c91e6e84a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:28.397240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:28.448997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3892a-8f66-4a36-93ad-ebc71d3f6435 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.455432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.457407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.457700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.458141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] PATH_INFO: `/v3/projects/7fe565b442cf41a5a9f89f77620a2d25/users/d6efa87974404f8eb324de63753a3217/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.477996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe565b442cf41a5a9f89f77620a2d25, user_id=d6efa87974404f8eb324de63753a3217, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.484219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.485134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 806/1604] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:28.486577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.502009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9697935-da4d-407d-a8ef-30f31c035596 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.505517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 799/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/7fe565b442cf41a5a9f89f77620a2d25/users/d6efa87974404f8eb324de63753a3217/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.510118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.555774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:28.580996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:28.603853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.604964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.605533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.605874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] PATH_INFO: `/v3/projects/b291344e1dba44f8a8863dfe6e78dd7f/users/84513bba3d7e4f949f59276d82d8e507/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.618438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c99661-7534-4460-8454-0178adb09184 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.625126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.627138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.627632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.628057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] PATH_INFO: `/v3/projects/a16deb2e741f484e94c851cddd0cc952/users/0fe5a01615e5474486c05dc809dd125d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.635820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b291344e1dba44f8a8863dfe6e78dd7f, user_id=84513bba3d7e4f949f59276d82d8e507, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.639165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.651435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16deb2e741f484e94c851cddd0cc952, user_id=0fe5a01615e5474486c05dc809dd125d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.654673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e9901-cfc3-47d1-9224-767bfd1d74f5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.662354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 807/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/b291344e1dba44f8a8863dfe6e78dd7f/users/84513bba3d7e4f949f59276d82d8e507/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.663103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:28.679717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 800/1607] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/a16deb2e741f484e94c851cddd0cc952/users/0fe5a01615e5474486c05dc809dd125d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.681730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580322-dc91-4857-8be4-8b4322b9494d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.682579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580322-dc91-4857-8be4-8b4322b9494d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.683086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580322-dc91-4857-8be4-8b4322b9494d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.726519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bd580322-dc91-4857-8be4-8b4322b9494d None None] MFA Rules not processed for user `1758d6fed42543fc9ff569b7051cb3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:28.727932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bd580322-dc91-4857-8be4-8b4322b9494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:28.768566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9522-6324-4524-80a6-27df3a681faa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:28.830195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 801/1608] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:28.830195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145627-5c00-49bd-8f8e-877d2b217caa None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.830195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145627-5c00-49bd-8f8e-877d2b217caa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.830195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145627-5c00-49bd-8f8e-877d2b217caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.913558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:28.915818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.916173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.916909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] PATH_INFO: `/v3/projects/f14e4e69a2db416caeef8036b55805b6/users/17986e47d6a34cd785ff91f0a000f2b5/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.918269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a9145627-5c00-49bd-8f8e-877d2b217caa None None] MFA Rules not processed for user `d6efa87974404f8eb324de63753a3217`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:28.919545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a9145627-5c00-49bd-8f8e-877d2b217caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:28.940438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14e4e69a2db416caeef8036b55805b6, user_id=17986e47d6a34cd785ff91f0a000f2b5, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:28.948317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd07fd-0b8e-4048-bee8-9dd9cdaba88b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:28.977530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 808/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:28 2022] PUT /identity/v3/projects/f14e4e69a2db416caeef8036b55805b6/users/17986e47d6a34cd785ff91f0a000f2b5/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:28.979020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97573-247c-4002-bedb-52de574a3163 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.979326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97573-247c-4002-bedb-52de574a3163 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.979582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97573-247c-4002-bedb-52de574a3163 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:28.980867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 809/1610] 10.222.0.16 () {54 vars in 961 bytes} [Mon Jul 4 04:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:28.983922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed56e-103c-45a8-9646-cad5695bdd62 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:28.984203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed56e-103c-45a8-9646-cad5695bdd62 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:28.984479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed56e-103c-45a8-9646-cad5695bdd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.039012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-969ed56e-103c-45a8-9646-cad5695bdd62 None None] MFA Rules not processed for user `84513bba3d7e4f949f59276d82d8e507`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:29.051093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-969ed56e-103c-45a8-9646-cad5695bdd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:29.075074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 802/1611] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.076421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f457527-902d-42ec-9add-6ac9d522dd0b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.076715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f457527-902d-42ec-9add-6ac9d522dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.077000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f457527-902d-42ec-9add-6ac9d522dd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.127692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3f457527-902d-42ec-9add-6ac9d522dd0b None None] MFA Rules not processed for user `0fe5a01615e5474486c05dc809dd125d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:29.129589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3f457527-902d-42ec-9add-6ac9d522dd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:29.140319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:29.187766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:29.217134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 810/1612] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.222619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3782-929e-47a3-b915-935f360845ce None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.222926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3782-929e-47a3-b915-935f360845ce None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.223208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3782-929e-47a3-b915-935f360845ce None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.231550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 811/1613] 10.222.0.16 () {54 vars in 961 bytes} [Mon Jul 4 04:14:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:29.233599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.234558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.235439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.247095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 803/1614] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.248919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.249325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.249726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.270215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:29.271970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:29.289678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:29.293261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:29.388097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 812/1615] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.393762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.406100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f176ea-53ee-4b93-a9e4-4644088eda10 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.417626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.418551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.419093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.420170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.421125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:29.423297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94843e-eae4-40e1-ae64-6b5fada17175 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:29.440281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 804/1616] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.441819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.460336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 813/1617] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:29.460530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8711b90-40af-4604-a2b8-a636dfea889e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.462449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.462862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.463318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.490623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:29.492485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:29.591122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.591850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.592144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.592424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.593140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:29.593882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 814/1618] 10.222.0.16 () {58 vars in 1058 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:29.595316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:29.604516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.654769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 805/1619] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:29.655803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.702983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:29.703993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3241-5e54-461b-b4cf-dacdd11921fe None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.734651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:29.775662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2916-c1b0-4fab-bb4f-b3c15c4076e4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.784721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.785768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.786291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.786756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.795091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.795937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.796228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.796509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.863201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:29.865109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:29.880573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 815/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:29.882143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.895822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682e212-15ad-4124-a1e5-4e9b749359b1 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.901541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.902640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.903298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.903872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.904978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:29.909007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:29.938864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 816/1621] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/users => generated 292 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:29.940166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:29.949622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:29.952322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:29.955880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1cb1c-cf01-4aa2-94a6-9426b1911a71 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:29.962691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:29.963379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:29.963671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:29.964005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:29.977781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 806/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:29.978865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.012120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07805be9-8994-4636-89a9-2a47053e9c9d service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.019344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.020035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.020320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.020630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.022015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.024731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.068679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 807/1623] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:29 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:30.071110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.098779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30790b8-4ed0-4397-9abf-d08e2fd642ec admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.107075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.109271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.112416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.114615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.114925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.117479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.119165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.124350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.129002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 817/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:30.132415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.154864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 808/1625] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:30.156731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f274-b934-45ed-8d8b-713a95945214 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.157400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.169942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.170732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.171025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.171309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.174049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.176700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.187974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105fa5f-30b2-473a-808a-62f05a556502 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.203048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 818/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:30.204272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.227288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0b6b-f537-4adc-b288-ad1ca0cbfe76 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.227401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.227401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.227401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.229894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] PATH_INFO: `/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf/users/b316083993e046c38f2f2e55c4cdfbdf/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.235476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.238636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.240629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.240941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] PATH_INFO: `/v3/projects/ffd37208f76d41428cd8ffed07596498/users/e427cf9b6f0e4e4da2f4a3780a69ff59/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.279214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee0af1948f34537a98fdbd2c3c79ebf, user_id=b316083993e046c38f2f2e55c4cdfbdf, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.287554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.290741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd37208f76d41428cd8ffed07596498, user_id=e427cf9b6f0e4e4da2f4a3780a69ff59, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.293507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.318526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 809/1627] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:30 2022] PUT /identity/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf/users/b316083993e046c38f2f2e55c4cdfbdf/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:30.318806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 819/1628] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:30 2022] PUT /identity/v3/projects/ffd37208f76d41428cd8ffed07596498/users/e427cf9b6f0e4e4da2f4a3780a69ff59/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:30.322442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.322911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.358943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:30.472211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec63cc-73a9-4549-9ea2-8b5a6d9caeda admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.486762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.487566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.488403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.488403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.491429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.492123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385123f2-d5f0-425e-85b8-d4b75b9f5dde admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.495626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a4b8d-ea8a-4cf7-a381-ac6d3980936b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.501738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.504899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.505183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.505474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.506249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.510174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.534627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 810/1629] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:30.537722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b17d-8d41-4c6a-975a-8bb92453f483 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.538043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b17d-8d41-4c6a-975a-8bb92453f483 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.538332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b17d-8d41-4c6a-975a-8bb92453f483 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.540066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 820/1630] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:30.541096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.546305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 811/1631] 10.222.0.16 () {54 vars in 912 bytes} [Mon Jul 4 04:14:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:30.547170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b17d-8d41-4c6a-975a-8bb92453f483 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.582837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b17d-8d41-4c6a-975a-8bb92453f483 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.587179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34323c-3546-4154-90aa-21e287db3236 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.676988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.676988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.676988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.676988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] PATH_INFO: `/v3/projects/ffd37208f76d41428cd8ffed07596498/users/e427cf9b6f0e4e4da2f4a3780a69ff59/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.683566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd37208f76d41428cd8ffed07596498, user_id=e427cf9b6f0e4e4da2f4a3780a69ff59, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.683566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf289da4-a790-48b4-8211-d3eff36fcfa9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.683566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:30.684087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.684087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.684087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.684087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] PATH_INFO: `/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf/users/b316083993e046c38f2f2e55c4cdfbdf/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.684456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee0af1948f34537a98fdbd2c3c79ebf, user_id=b316083993e046c38f2f2e55c4cdfbdf, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:30.684456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c976a-fe2f-4bbf-a329-4d726ee928e8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:30.684613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 812/1632] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:30 2022] PUT /identity/v3/projects/ffd37208f76d41428cd8ffed07596498/users/e427cf9b6f0e4e4da2f4a3780a69ff59/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:30.684613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd328-0506-4153-bc7d-b00e9493aa57 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.684613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd328-0506-4153-bc7d-b00e9493aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.684613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd328-0506-4153-bc7d-b00e9493aa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.684890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 821/1633] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:30 2022] PUT /identity/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf/users/b316083993e046c38f2f2e55c4cdfbdf/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:30.695201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.695612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.695878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.699553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-eedfd328-0506-4153-bc7d-b00e9493aa57 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:30.704994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eedfd328-0506-4153-bc7d-b00e9493aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:30.752705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] MFA Rules not processed for user `e427cf9b6f0e4e4da2f4a3780a69ff59`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:30.773153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:30.885617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 813/1634] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:30.888957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.889265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.889531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.898566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 822/1635] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:30.900237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:30.924496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f70866-bad2-4a26-afcb-a8367204e3e6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:30.931226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:30.932176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:30.932624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:30.933190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:30.947807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] MFA Rules not processed for user `b316083993e046c38f2f2e55c4cdfbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:30.949030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:31.006188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:31.008176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:31.026313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 823/1636] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:31.027351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:31.063651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 814/1637] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:31.064717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:31.074984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:31.128117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d514f-f2ba-4c2e-ab2b-dce00380626d service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:31.133852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:31.134668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.136517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.137027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.165848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:31.169710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ab746-cc7e-48c3-85e1-bdfb2eb1517f service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:31.205891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 824/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:31.209963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da79fa-2e15-4af7-8d6e-f4f8177744aa None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.210361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da79fa-2e15-4af7-8d6e-f4f8177744aa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.212599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da79fa-2e15-4af7-8d6e-f4f8177744aa None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.216549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 825/1639] 10.222.0.16 () {54 vars in 912 bytes} [Mon Jul 4 04:14:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:31.224961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.224961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.224961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.227660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663bc77-40fd-468c-976e-7285ed1400bf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:31.232180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:31.232859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.233150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.233431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.264512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:31.268291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:31.268421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:31.270296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:31.304638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 815/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:31.335397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 826/1641] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:14:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:14:31.449450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:31.459107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168007-0788-4f3b-a5b1-9c33155ebbb6 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:31.464974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:31.466117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.466676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.467846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.497148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:31.499793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:31.524347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:31.561790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 816/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:31.627312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:31.644755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f71587-88c8-439f-8693-d9d8cb18d8c3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:31.651634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:31.659751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:31.660229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:31.660688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:31.694643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:31.696477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:31.743570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 827/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:33.439955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:33.550772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf39ad-12b4-4c72-8b39-d537b0657ad4 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:33.558044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:33.561889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:33.562249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:33.562629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] PATH_INFO: `/v3/users/1758d6fed42543fc9ff569b7051cb3e8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:33.571348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1758d6fed42543fc9ff569b7051cb3e8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:33.572321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: WARNING py.warnings [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:33.572321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: warnings.warn(msg) Jul 04 04:14:33.572321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: Jul 04 04:14:33.573670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:33.684705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1758d6fed42543fc9ff569b7051cb3e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:33.742389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1758d6fed42543fc9ff569b7051cb3e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:33.759094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1758d6fed42543fc9ff569b7051cb3e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:33.821065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1758d6fed42543fc9ff569b7051cb3e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:33.821065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-3d19b52a-afd2-470c-ae2f-10b04032ba5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1758d6fed42543fc9ff569b7051cb3e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:33.821499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:33.823301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 817/1644] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:33 2022] DELETE /identity/v3/users/1758d6fed42543fc9ff569b7051cb3e8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:33.829650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:33.829650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:33.829650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:33.841291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea10a-1253-4221-8eaa-b42a1de94ada service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:33.914617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:33.916340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:33.995111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:33.997186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:33.997896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:33.998598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] PATH_INFO: `/v3/users/d6efa87974404f8eb324de63753a3217` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:34.000369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6efa87974404f8eb324de63753a3217)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:34.003421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:34.143397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6efa87974404f8eb324de63753a3217'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.147582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 818/1645] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:34.148485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:34.157129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.167562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6efa87974404f8eb324de63753a3217'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.170317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6efa87974404f8eb324de63753a3217'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.177666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86c0e5-8a42-42af-b246-56ecead8c1c2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:34.191257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:34.192183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:34.192475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:34.192741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] PATH_INFO: `/v3/users/84513bba3d7e4f949f59276d82d8e507` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:34.195129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84513bba3d7e4f949f59276d82d8e507)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:34.199135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6efa87974404f8eb324de63753a3217'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.207797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:34.241186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6efa87974404f8eb324de63753a3217'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.253393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:34.263365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 828/1646] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:33 2022] DELETE /identity/v3/users/d6efa87974404f8eb324de63753a3217 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:14:34.269982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:34.276774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.370829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84513bba3d7e4f949f59276d82d8e507'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.436430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:34.461940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84513bba3d7e4f949f59276d82d8e507'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.495572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84513bba3d7e4f949f59276d82d8e507'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.526057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee1646-08fe-4e5a-8a51-1954ddc62c20 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:34.527724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84513bba3d7e4f949f59276d82d8e507'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.532562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6b03bedf-cfc6-459f-8621-cff62c68f3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84513bba3d7e4f949f59276d82d8e507'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:34.540621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:34.542834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:34.543138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:34.543421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:34.548603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 819/1647] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:34 2022] DELETE /identity/v3/users/84513bba3d7e4f949f59276d82d8e507 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:34.551881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.552630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a3cc-a561-402a-9100-e0588d79311c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:34.552931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a3cc-a561-402a-9100-e0588d79311c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:34.553215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a3cc-a561-402a-9100-e0588d79311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:34.604643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9550a3cc-a561-402a-9100-e0588d79311c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:34.606778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9550a3cc-a561-402a-9100-e0588d79311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.757870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:34.759829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:34.805412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 820/1648] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:34.808455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:34.808753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:34.809036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:34.815427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 829/1649] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:34.832171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:34.833850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.855937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388134-c13c-4375-9572-3b8bcb695823 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:34.862861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:34.870212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:34.893682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 821/1650] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:34.958150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:34.959664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:35.010496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:35.010897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:35.011164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:35.012185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:35.046544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:35.048002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:35.070571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 830/1651] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:14:35.089597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f13d0-28e5-4498-9a2b-c80fcff2328a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:35.096640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:35.098011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:35.098478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:35.098763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:35.115480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:35.117159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:35.150409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 822/1652] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:36.291652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:36.298010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:36.334059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:36.336411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:36.340739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401b2ea-391d-4a9f-9c09-e25ea3f323dc service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:36.350894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:36.356744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:36.357251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:36.357748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] PATH_INFO: `/v3/users/c756a1bc870a4ec5a5b629c91e6e84a3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:36.359664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c756a1bc870a4ec5a5b629c91e6e84a3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:36.361909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:36.366601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c532ca-5b49-4bf0-b644-99e590345432 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:36.390491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:36.393724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:36.394066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:36.394802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] PATH_INFO: `/v3/projects/7fe565b442cf41a5a9f89f77620a2d25` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:36.397762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe565b442cf41a5a9f89f77620a2d25)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:36.399898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:36.509273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c756a1bc870a4ec5a5b629c91e6e84a3'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.527026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c756a1bc870a4ec5a5b629c91e6e84a3'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.527736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:36.529012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c756a1bc870a4ec5a5b629c91e6e84a3'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.543562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c756a1bc870a4ec5a5b629c91e6e84a3'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.559193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:36.575088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d6613c81-d81a-4eb2-aedc-033032529b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c756a1bc870a4ec5a5b629c91e6e84a3'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.577536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe565b442cf41a5a9f89f77620a2d25', 'request_id': 'req-5ca559ca-d066-4784-9b99-3374128c4bd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.589340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 831/1653] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:36 2022] DELETE /identity/v3/users/c756a1bc870a4ec5a5b629c91e6e84a3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:36.593390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:36.593658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe565b442cf41a5a9f89f77620a2d25', 'request_id': 'req-5ca559ca-d066-4784-9b99-3374128c4bd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.594019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:36.594630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:36.611889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe565b442cf41a5a9f89f77620a2d25', 'request_id': 'req-5ca559ca-d066-4784-9b99-3374128c4bd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.631307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:36.632706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:36.639322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] The token cache is being invalidate because project 7fe565b442cf41a5a9f89f77620a2d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:36.639605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.641655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.649957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 823/1654] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:36 2022] DELETE /identity/v3/projects/7fe565b442cf41a5a9f89f77620a2d25 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:36.651348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:36.652848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:36.756398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 832/1655] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:36.780215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:36.795065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:36.839887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca559ca-d066-4784-9b99-3374128c4bd6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:36.847789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:36.849089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:36.849756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:36.850300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] PATH_INFO: `/v3/users/b316083993e046c38f2f2e55c4cdfbdf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:36.853332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b316083993e046c38f2f2e55c4cdfbdf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:36.856105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:36.954204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b316083993e046c38f2f2e55c4cdfbdf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:36.995009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4ec75-8369-4bea-bb6c-6f5010cf460e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:36.997806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b316083993e046c38f2f2e55c4cdfbdf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.008376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:37.009116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.009428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.009729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.028357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b316083993e046c38f2f2e55c4cdfbdf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.063457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b316083993e046c38f2f2e55c4cdfbdf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.064988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bb04b62b-dcc8-47c0-8792-ee09a0abc3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b316083993e046c38f2f2e55c4cdfbdf'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.065930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:37.070115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:37.083858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 824/1656] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:36 2022] DELETE /identity/v3/users/b316083993e046c38f2f2e55c4cdfbdf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:14:37.084246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.084246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.097881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.117935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 833/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:37.120755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:37.124218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.191869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0ae3-0ebc-489b-ba80-8b07e6145f61 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:37.220546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] MFA Rules not processed for user `dca54bc220104e79aa727f30399f3411`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:37.222099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.272920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:37.362886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:37.364884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.365369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.365976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] PATH_INFO: `/v3/projects/94810165aeb641848b753c96ca3ad7a8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.376526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94810165aeb641848b753c96ca3ad7a8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:37.377746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: WARNING py.warnings [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 04:14:37.377746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: warnings.warn(msg) Jul 04 04:14:37.377746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: Jul 04 04:14:37.390693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:37.408700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 825/1658] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:37.409867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:37.414178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.448690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e609086-d53e-4a1b-882c-f28f52a0ecfd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:37.459861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:37.460767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.461053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.461334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] PATH_INFO: `/v3/projects/b291344e1dba44f8a8863dfe6e78dd7f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.465807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b291344e1dba44f8a8863dfe6e78dd7f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:37.482958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:37.539486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:37.584594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94810165aeb641848b753c96ca3ad7a8', 'request_id': 'req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.602874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94810165aeb641848b753c96ca3ad7a8', 'request_id': 'req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.607467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b291344e1dba44f8a8863dfe6e78dd7f', 'request_id': 'req-911be10e-f8a5-435b-a8dc-2f444a5325b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.618004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94810165aeb641848b753c96ca3ad7a8', 'request_id': 'req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.624003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b291344e1dba44f8a8863dfe6e78dd7f', 'request_id': 'req-911be10e-f8a5-435b-a8dc-2f444a5325b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.652092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b291344e1dba44f8a8863dfe6e78dd7f', 'request_id': 'req-911be10e-f8a5-435b-a8dc-2f444a5325b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.656530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] The token cache is being invalidate because project 94810165aeb641848b753c96ca3ad7a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:37.657982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.660014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3d0b1eed-1c1d-45cd-892b-ca861dd457a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.663094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 834/1659] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:37 2022] DELETE /identity/v3/projects/94810165aeb641848b753c96ca3ad7a8 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:37.665574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.668526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.669732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.675743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] The token cache is being invalidate because project b291344e1dba44f8a8863dfe6e78dd7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:37.676833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.682303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-911be10e-f8a5-435b-a8dc-2f444a5325b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:37.685299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 826/1660] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:37 2022] DELETE /identity/v3/projects/b291344e1dba44f8a8863dfe6e78dd7f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:37.689948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.690848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.692576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.713188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] MFA Rules not processed for user `dca54bc220104e79aa727f30399f3411`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:37.716133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.731861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:37.738670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.856798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 835/1661] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:37.857841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:37.862346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.871016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 827/1662] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:37.880648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:37.884907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:37.893445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7682a-b529-4570-84c1-478f6df3d9c3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:37.910229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:37.912359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:37.912822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:37.913287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] PATH_INFO: `/v3/users/e427cf9b6f0e4e4da2f4a3780a69ff59` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:37.916603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e427cf9b6f0e4e4da2f4a3780a69ff59)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:37.923299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:37.991719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:38.053985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e427cf9b6f0e4e4da2f4a3780a69ff59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:38.063668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc7ff7-c647-4201-b191-59eedd34fb7a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.072980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.074325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e427cf9b6f0e4e4da2f4a3780a69ff59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:38.074808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e427cf9b6f0e4e4da2f4a3780a69ff59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:38.075363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.075696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.075969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.081652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:38.090024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e427cf9b6f0e4e4da2f4a3780a69ff59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:38.131895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.133543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971eda0-f5a2-4c7f-84aa-b735e023cc83 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.150816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e427cf9b6f0e4e4da2f4a3780a69ff59'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:38.164260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 828/1663] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:38.165387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.165675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.165959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.167333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 836/1664] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:37 2022] DELETE /identity/v3/users/e427cf9b6f0e4e4da2f4a3780a69ff59 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:38.209465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:38.213598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:38.383738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 829/1665] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:38.388100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.399140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34274eb-f162-4057-9308-c402946dbdaf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.405731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.416027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.416027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.416027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.416651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:38.416971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.418869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.437277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 837/1666] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:38.439544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.451735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35933949-e89f-4fa8-8f2b-7a73accb82ac None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.460873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.461629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.462112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.463838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.464650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.466896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.539211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 830/1667] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:38 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:14:38.543243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.555847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e2b5-8d5e-45c0-af6e-2cb6edbc3ee9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.622429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:38.688616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.689332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.689619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.689898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.692635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.694798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.732723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 838/1668] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:38 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:38.737834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.758649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.763289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:38.764336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3d76e-ab85-4681-bf42-82416b6e64ce admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.781664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.782624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.782926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.785021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.786846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.794096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.806949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8478c3-fe0d-4d8c-899c-ff8cabd78f42 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.834973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.837804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.838119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.838424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] PATH_INFO: `/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.843685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee0af1948f34537a98fdbd2c3c79ebf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.845211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 831/1669] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:38.847266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.850250 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.879559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50876f16-e2ac-491a-abae-da83cf0fa78c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:38.895996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:38.897627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:38.899087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:38.899835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:38.900605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:38.905454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:38.942345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 832/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:38.949343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:38.999907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d2a6-4aac-4db8-8c33-64d42479753b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:39.036420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee0af1948f34537a98fdbd2c3c79ebf', 'request_id': 'req-adafb178-26d1-4f97-808b-56534541cbbe'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.067151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee0af1948f34537a98fdbd2c3c79ebf', 'request_id': 'req-adafb178-26d1-4f97-808b-56534541cbbe'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.093332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee0af1948f34537a98fdbd2c3c79ebf', 'request_id': 'req-adafb178-26d1-4f97-808b-56534541cbbe'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.115561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] The token cache is being invalidate because project cee0af1948f34537a98fdbd2c3c79ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:39.115872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.116618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.122824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 839/1671] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:38 2022] DELETE /identity/v3/projects/cee0af1948f34537a98fdbd2c3c79ebf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:14:39.123344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:39.132838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:39.234033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:39.235021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.235312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.235592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] PATH_INFO: `/v3/projects/1fe6dc98d0824996b0e322503e94327b/users/51ef62d7e0a940c588c1fa61ea77681e/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.277385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe6dc98d0824996b0e322503e94327b, user_id=51ef62d7e0a940c588c1fa61ea77681e, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:39.281351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ae71a-06d2-425c-ba20-c3daa49fb3a5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:39.318232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 833/1672] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:38 2022] PUT /identity/v3/projects/1fe6dc98d0824996b0e322503e94327b/users/51ef62d7e0a940c588c1fa61ea77681e/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:39.319669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc217-1f1b-4b49-af2e-6e5de3b96123 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.319963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc217-1f1b-4b49-af2e-6e5de3b96123 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.323574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc217-1f1b-4b49-af2e-6e5de3b96123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.351759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafb178-26d1-4f97-808b-56534541cbbe admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:39.384532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-491dc217-1f1b-4b49-af2e-6e5de3b96123 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:39.386852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-491dc217-1f1b-4b49-af2e-6e5de3b96123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:39.400229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:39.483516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:39.523816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:39.524733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.526138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.526463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] PATH_INFO: `/v3/projects/16c2e6487a1b435c9a362c1fd52b9978` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.529339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c2e6487a1b435c9a362c1fd52b9978)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:39.535261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:39.551514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 834/1673] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:39.552663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d469e48-f982-4db1-8653-ca731f39c8da None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.552963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d469e48-f982-4db1-8653-ca731f39c8da None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.553255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d469e48-f982-4db1-8653-ca731f39c8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.646741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6d469e48-f982-4db1-8653-ca731f39c8da None None] MFA Rules not processed for user `4041faa2919d4760ae10907070a10dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:39.648408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6d469e48-f982-4db1-8653-ca731f39c8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:39.697084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c2e6487a1b435c9a362c1fd52b9978', 'request_id': 'req-3c47444a-82f5-495e-a272-700bb06ede07'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.708078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c2e6487a1b435c9a362c1fd52b9978', 'request_id': 'req-3c47444a-82f5-495e-a272-700bb06ede07'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.748719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c2e6487a1b435c9a362c1fd52b9978', 'request_id': 'req-3c47444a-82f5-495e-a272-700bb06ede07'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.783001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 835/1674] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:39.785043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72709252-083e-49ee-99a1-de90fee97bab None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.785346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72709252-083e-49ee-99a1-de90fee97bab None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.785631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72709252-083e-49ee-99a1-de90fee97bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.791660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] The token cache is being invalidate because project 16c2e6487a1b435c9a362c1fd52b9978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:39.791958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.800819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c47444a-82f5-495e-a272-700bb06ede07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:39.800819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 840/1675] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:39 2022] DELETE /identity/v3/projects/16c2e6487a1b435c9a362c1fd52b9978 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:39.801461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf0f2-8b5b-40fa-81a6-f6c88a00f8b5 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.801461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf0f2-8b5b-40fa-81a6-f6c88a00f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.801741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf0f2-8b5b-40fa-81a6-f6c88a00f8b5 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.803291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 841/1676] 10.222.0.16 () {54 vars in 912 bytes} [Mon Jul 4 04:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:39.804352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c0da-a410-407a-9658-27d36e49cfbf None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.804634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c0da-a410-407a-9658-27d36e49cfbf None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.804912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c0da-a410-407a-9658-27d36e49cfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:39.833837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3171c0da-a410-407a-9658-27d36e49cfbf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:39.851273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3171c0da-a410-407a-9658-27d36e49cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:39.855579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-72709252-083e-49ee-99a1-de90fee97bab None None] MFA Rules not processed for user `51ef62d7e0a940c588c1fa61ea77681e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:39.857622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-72709252-083e-49ee-99a1-de90fee97bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:39.990407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 836/1677] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:39.991287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:39.991406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:39.991683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.043033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:40.053272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.059393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 842/1678] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:40.059793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.060091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.061903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.071682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 837/1679] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:40.074287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.079243 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.133854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:40.145591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:40.146921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.248438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73659-9fdd-4c22-a39d-e39012400751 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.250151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 843/1680] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:40.251180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.252908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.272439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad701b-161f-44f3-abff-8abd2b32ec89 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.276946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.277849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.278122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.278594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.278847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.278943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.279131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.279470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.280326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.281924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.282391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16ada3-174f-40b0-98e6-3f24fd052ce1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.303615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:40.317755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.320393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.360254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 838/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:40.361287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.364545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 844/1682] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:40 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:40.364545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.364545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.364545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.392409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc8ab-8e69-47f2-87c4-2083f7d75482 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.393126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:40.394958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.549444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.550161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.550471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.550751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.551915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:40.552493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.553746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.573212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 839/1683] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:40.573212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.573470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.574770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 845/1684] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:40.575712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.583984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.597171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c69319-cc89-41b4-85ed-e63078fe36d7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.607968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.609437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.609856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.610141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] PATH_INFO: `/v3/projects/ffd37208f76d41428cd8ffed07596498` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.611955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd37208f76d41428cd8ffed07596498)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.614602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.717809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd37208f76d41428cd8ffed07596498', 'request_id': 'req-8ceeba6e-876e-4622-b54b-5ed61151c73e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:40.731345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd37208f76d41428cd8ffed07596498', 'request_id': 'req-8ceeba6e-876e-4622-b54b-5ed61151c73e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:40.740316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd37208f76d41428cd8ffed07596498', 'request_id': 'req-8ceeba6e-876e-4622-b54b-5ed61151c73e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:40.749902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a084379-a0be-414b-8d63-00f34e447e8d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.769559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] The token cache is being invalidate because project ffd37208f76d41428cd8ffed07596498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:40.771333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:40.774196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:40.781280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 840/1685] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:40 2022] DELETE /identity/v3/projects/ffd37208f76d41428cd8ffed07596498 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:40.782427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.785113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.861994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:40.911411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.912361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.912792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.913222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.921894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:40.943427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.945164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.953542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeba6e-876e-4622-b54b-5ed61151c73e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:40.969737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 846/1686] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:40.973124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:40.973904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:40.974221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:40.974527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:40.975465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:40.976997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:40.977707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:40.982472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:41.010807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe675-9514-4513-8af1-564fa5e87aa2 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.017719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.018800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.019270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.019954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.021121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.023422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cf768-9cd4-493d-8799-14fd36df74f4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.035697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 841/1687] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:40 2022] POST /identity/v3/users => generated 309 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.043527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.063105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:41.088899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 847/1688] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:40 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.095995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.096313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.096605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.104942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d912289-0b6e-4fd6-8fef-8e59272b3733 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.144997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:41.146494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:41.204811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:41.260157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.261285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.262633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.265850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.266835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:41.271403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 848/1689] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:41.273152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.275106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.277080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.292492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1d783-d9e0-47c8-bafc-8a601c392444 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.300736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 842/1690] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:41.301904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.303112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.304467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.305011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.306756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.307973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.312879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3dd8f-e5f4-49f2-9f6f-b31b2c091238 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.325094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9708f-eaa2-41b8-9d93-2d82545a7c49 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.335166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 849/1691] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:41.337718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.338884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.339663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.340042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.340837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.340837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.341285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.342132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.344882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.374013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:41.375390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:41.388105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 843/1692] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/users => generated 283 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.389410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.403054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 850/1693] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:41.403054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.415350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08358f8-1fa5-4dd5-9ff8-f153ba070a72 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.421729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b009d7-4236-4fb5-9b7b-4a202d4abf1b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.429475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.442190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.442661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.442998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.443788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.457437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.460525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.461187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.461505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.461784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] PATH_INFO: `/v3/projects/fb781a08c1a741c6b84fccf5af550859/users/30721e546b21443f9348636e8ee0fa7c/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.480107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb781a08c1a741c6b84fccf5af550859, user_id=30721e546b21443f9348636e8ee0fa7c, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.496974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.523590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 844/1694] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.524535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.526360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 851/1695] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:41 2022] PUT /identity/v3/projects/fb781a08c1a741c6b84fccf5af550859/users/30721e546b21443f9348636e8ee0fa7c/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:41.527573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.698681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63043067-4923-4213-80e7-6a8266f3b54e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.707432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8108b-0cd6-4b7f-8029-89898e900df3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.715014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.716788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.717295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.717897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.718914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.720879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.721272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.726393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.726393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.726393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.726393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:41.728059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.729866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.747880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 845/1696] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:41.749071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.766831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 852/1697] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:41.766831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.769710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:41.778491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a5765-c460-459c-8f71-3f052d74bf83 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.792396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.793185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.793537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.793881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.794782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.799415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.804250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3f1ba-9d7f-4149-aaaa-65d0a93d3d86 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.817696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.818890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.819396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.819860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.831836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.835798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.859824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 846/1698] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.860741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.867698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 853/1699] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:41.869299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.886117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012e0ba-7129-4a38-a4fc-d7d6454e164a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.890269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadfb5f-ce39-4ede-8c77-278a26eafeb4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.898134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.899599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.899887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.900166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.900939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.902632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.905223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.905923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.906202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.906518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.907241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:41.910979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:41.931754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 847/1700] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:41.931754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.978105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0f05-d977-46ca-8d42-ddfeb5750405 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:41.987055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 854/1701] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:41.987055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:41.996119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:41.997084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:41.997379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:41.997660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:41.998499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.000133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.033460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 848/1702] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:42.036747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:42.059298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b6dc-1c12-4656-b810-049fc666eb1e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.110166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:42.139224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4d046-c51c-4cdb-b4cc-aef1ecd9140a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.149115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.154569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.154903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.155752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] PATH_INFO: `/v3/projects/fb781a08c1a741c6b84fccf5af550859/users/30721e546b21443f9348636e8ee0fa7c/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.161735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb781a08c1a741c6b84fccf5af550859, user_id=30721e546b21443f9348636e8ee0fa7c, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.166946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.193803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.197313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 855/1703] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:41 2022] PUT /identity/v3/projects/fb781a08c1a741c6b84fccf5af550859/users/30721e546b21443f9348636e8ee0fa7c/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:42.198111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:42.198652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.199357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.199357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] PATH_INFO: `/v3/projects/3aff6e1ced2e4a90b83656f66a4ee05b/users/72737ccdbc164906a903f58d0ea660df/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.242215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aff6e1ced2e4a90b83656f66a4ee05b, user_id=72737ccdbc164906a903f58d0ea660df, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.250917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.278845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 849/1704] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:42 2022] PUT /identity/v3/projects/3aff6e1ced2e4a90b83656f66a4ee05b/users/72737ccdbc164906a903f58d0ea660df/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:42.280730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:42.378487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26df757-4b4c-4472-98e7-1d65e821556c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.444926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65c3e9-623e-40c0-ba39-4478ab3695f1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.455929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.456888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.457178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.457459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.459575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.461220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1d520-9355-48cf-b673-f1a8a360f1fc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.493177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 850/1705] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:42.494880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.496421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.496884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.514301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.515095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.516493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.519011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.520141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.526149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5c891-b98f-4bc0-8cc4-b92feb353582 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.582309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] MFA Rules not processed for user `30721e546b21443f9348636e8ee0fa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:42.588417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 856/1706] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:42 2022] POST /identity/v3/users => generated 288 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:42.588638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:42.589677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.590083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.590928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.678979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] MFA Rules not processed for user `72737ccdbc164906a903f58d0ea660df`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:42.681531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:42.699162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:42.824513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 851/1707] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:42.824513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:42.849685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04fdd1-6b3e-4ac6-81a1-d6b8a720c9c3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.865783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.868099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.868544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.869018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] PATH_INFO: `/v3/projects/cd464d738d4c451ba7a629970d36917b/users/1da1b96e033d4a84941c8a26752973ce/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.906779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 857/1708] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:42.910558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:42.930532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd464d738d4c451ba7a629970d36917b, user_id=1da1b96e033d4a84941c8a26752973ce, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.939693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfd612-1df0-4c5f-b934-a6f2104a7d74 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:42.940995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.966024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:42.967827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:42.969403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:42.973106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:42.978522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:42.984956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:42.995323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:42.995896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 852/1709] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:42 2022] PUT /identity/v3/projects/cd464d738d4c451ba7a629970d36917b/users/1da1b96e033d4a84941c8a26752973ce/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:42.999983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:43.009621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:43.027486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 858/1710] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:43.028934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:43.059914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665f6-608d-4988-975a-773d2a259538 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:43.256430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:43.257138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.257431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.257708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.258478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:43.260593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f6937-d40b-4f49-ac82-92cceda60b58 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:43.266546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a150-00d8-4e94-85e5-47d56435d153 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:43.272684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:43.274540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.276337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.276572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.382971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 859/1711] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:43 2022] POST /identity/v3/projects => generated 363 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:43.385356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.385649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.386126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.478143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] MFA Rules not processed for user `1da1b96e033d4a84941c8a26752973ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:43.479404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:43.523780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:43.525442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:43.549767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 853/1712] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:43.552349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:43.582755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1d3a7-b443-4b61-b2f5-d60881be5d84 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:43.718061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:43.737154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:43.738168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.738700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.739264 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.740273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:43.742193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91412092-b929-46fc-bbad-b8c2c9f5b384 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:43.757606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 860/1713] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:43.758038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:43.792751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 854/1714] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:43.794600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f96fa-f154-4ff1-951a-86157cf253d1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:43.805342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:43.809830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4ce73-df08-411d-bdc3-8960d903881f None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.809830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4ce73-df08-411d-bdc3-8960d903881f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.810680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.811076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.811468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.813627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:43.815843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf25046-97d7-48e0-9d6f-e53346a399d3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:43.816282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4ce73-df08-411d-bdc3-8960d903881f None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.823043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 855/1715] 10.222.0.16 () {54 vars in 960 bytes} [Mon Jul 4 04:14:43 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:43.825643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4ce73-df08-411d-bdc3-8960d903881f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:43.859892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4ce73-df08-411d-bdc3-8960d903881f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:43.865577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:43.875054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 861/1716] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:43 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:43.880485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.881164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52910741-0ec2-4098-a702-c859ba859a4c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:43.881556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.881908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52910741-0ec2-4098-a702-c859ba859a4c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:43.882247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.882630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52910741-0ec2-4098-a702-c859ba859a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:43.885346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:43.887651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:43.928845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-52910741-0ec2-4098-a702-c859ba859a4c None None] MFA Rules not processed for user `dca54bc220104e79aa727f30399f3411`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:43.933832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-52910741-0ec2-4098-a702-c859ba859a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:43.977365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 856/1717] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:43 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:43.978236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:44.098690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:44.132826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 862/1718] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:44.134887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.134990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.135204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.176444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b88e79-eb8d-44de-b712-344293a973a9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:44.186495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:44.191355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:44.192989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.193276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.193561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] PATH_INFO: `/v3/projects/9942552034f44a88a79191475ad6d4d4/users/7fececcc55844fe1beb2db7e454342c5/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.194302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:44.235768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9942552034f44a88a79191475ad6d4d4, user_id=7fececcc55844fe1beb2db7e454342c5, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:44.239412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:44.273390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 857/1719] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:43 2022] PUT /identity/v3/projects/9942552034f44a88a79191475ad6d4d4/users/7fececcc55844fe1beb2db7e454342c5/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:44.274295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:44.361101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821461ec-6212-4722-8f45-b86b6464f290 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:44.367076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:44.368144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.368523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.369135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] PATH_INFO: `/v3/users/51ef62d7e0a940c588c1fa61ea77681e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.371106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ef62d7e0a940c588c1fa61ea77681e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:44.373401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:44.430398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 863/1720] 10.222.0.16 () {58 vars in 1057 bytes} [Mon Jul 4 04:14:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:44.435528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:44.486168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c84755-c818-4f61-96ed-f21b565d7f25 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:44.494604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ef62d7e0a940c588c1fa61ea77681e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:44.520788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:44.523797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:44.524702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.526237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.526609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.532385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:44.537904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ef62d7e0a940c588c1fa61ea77681e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:44.549928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525429c-da38-4a1c-966b-49713ed53c67 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:44.568253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ef62d7e0a940c588c1fa61ea77681e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:44.588268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ef62d7e0a940c588c1fa61ea77681e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:44.591633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ef62d7e0a940c588c1fa61ea77681e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:44.596115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 864/1721] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:44.597888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.598190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.602875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.619589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 858/1722] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:44 2022] DELETE /identity/v3/users/51ef62d7e0a940c588c1fa61ea77681e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:44.622461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:44.625061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:44.668044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] MFA Rules not processed for user `4041faa2919d4760ae10907070a10dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:44.669482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:44.829159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad993-b68e-4429-8df5-56b7278e099c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:44.839050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:44.839754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.840033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.840310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.841007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:44.847594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd68da-635b-4656-a01c-31b8765579f4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:44.891421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 859/1723] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:44 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:44.892472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:44.892764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:44.893045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:44.919014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 865/1724] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:44.920178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:44.922789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:44.969760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] MFA Rules not processed for user `7fececcc55844fe1beb2db7e454342c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:44.971185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.169850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ec6e-4ddc-4334-8e20-1167daf58107 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.182098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.182999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.183542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.183925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.185629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 860/1725] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:45.186884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.187379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.191999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.219447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.221083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa8ee2-8bcf-42b9-8851-921998806714 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.224830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860bcf5-46f4-4877-a9af-5a30b63cb590 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.238876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.239821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.240109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.240391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] PATH_INFO: `/v3/projects/7fc47b0c34044e199788455b83bcb029/users/226d208d7c8d4e999f245ac9ada88a9d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.255492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 866/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:45.257137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.257323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.257586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.284186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc47b0c34044e199788455b83bcb029, user_id=226d208d7c8d4e999f245ac9ada88a9d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.287514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41c65f-0177-4b11-b56b-565256e7d4e2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.321046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 861/1727] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:45 2022] PUT /identity/v3/projects/7fc47b0c34044e199788455b83bcb029/users/226d208d7c8d4e999f245ac9ada88a9d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:45.324036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.324332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.324615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.325160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] MFA Rules not processed for user `17986e47d6a34cd785ff91f0a000f2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:45.327003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.337368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:45.365279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:45.366857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.422504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:45.467188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 867/1728] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:45.467188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.506564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 862/1729] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:45.506564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.506564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.523832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e07b-4866-42c6-a962-07772e3cf09a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.529445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.530237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.530899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.531327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.532133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.534324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03b659-6de6-4cd8-b648-2ad28ddd35e2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.593214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 863/1730] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:45.595272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42593-c78d-424f-a0a7-6b169f318bc8 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.597045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42593-c78d-424f-a0a7-6b169f318bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.597410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42593-c78d-424f-a0a7-6b169f318bc8 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.606896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf692ae0-f2a3-48aa-9160-aac20b3786c6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.615956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 864/1731] 10.222.0.16 () {54 vars in 912 bytes} [Mon Jul 4 04:14:45 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:14:45.617035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42593-c78d-424f-a0a7-6b169f318bc8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.716968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.717813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.718058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.718526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.719104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.723684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547feb1-6275-4868-81e2-357bc96e8e5d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.742016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 868/1732] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:45.744624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.744999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.745349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.769052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42593-c78d-424f-a0a7-6b169f318bc8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.773770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] MFA Rules not processed for user `17986e47d6a34cd785ff91f0a000f2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:45.776480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.780851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.781594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.782038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.782331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.783112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.791827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.817089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 865/1733] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:45.818077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.835367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:45.853211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:45.874816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 869/1734] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:45.890422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:45.904018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7a862-76b5-4424-9f86-bfb3a8b04c29 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.910954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.911790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.912170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.912561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.913363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.917399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a98706-c84e-4ef9-8dd9-df257693e475 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.929272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1d2b1-bf88-4996-a6be-47dbdfe11dc8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:45.933638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:45.934600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.935035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.935455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.958494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:45.960277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 870/1735] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/users => generated 294 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:45.961564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:45.962148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:45.962630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:45.963086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:45.988841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 866/1736] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:45.992206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.003667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:46.005161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:46.018768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad48b14-ae37-415a-8a62-d136f6307202 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.022983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 871/1737] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:14:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:46.024083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.032204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.033267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.033546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.033826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] PATH_INFO: `/v3/projects/ac16eee1a511464b8bfc17c09a885306/users/b5b1cdd3671c46bba80d3f106383d54e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.051235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf332d-a945-42e9-a4e0-c757366c0154 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.056793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.057976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.058324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.058694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] PATH_INFO: `/v3/projects/7fc47b0c34044e199788455b83bcb029/users/226d208d7c8d4e999f245ac9ada88a9d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.060909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc47b0c34044e199788455b83bcb029, user_id=226d208d7c8d4e999f245ac9ada88a9d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:46.063837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:46.079003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac16eee1a511464b8bfc17c09a885306, user_id=b5b1cdd3671c46bba80d3f106383d54e, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:46.082698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:46.100875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 872/1738] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:46 2022] PUT /identity/v3/projects/7fc47b0c34044e199788455b83bcb029/users/226d208d7c8d4e999f245ac9ada88a9d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:46.100875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.120486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 867/1739] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:45 2022] PUT /identity/v3/projects/ac16eee1a511464b8bfc17c09a885306/users/b5b1cdd3671c46bba80d3f106383d54e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:46.123833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.168945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:46.254238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc412b5-a652-4579-9458-fab2879576ca admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.259698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.260365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.260365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.261412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.287889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:46.289523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90290a3-b24f-4321-a807-2d6dfc957741 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:46.314737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 868/1740] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:46.315943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.316277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.321720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd726d-7043-4a0c-80ea-8dee338b7c03 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.327237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.385117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] MFA Rules not processed for user `226d208d7c8d4e999f245ac9ada88a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:46.408778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:46.640896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 869/1741] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:46.641781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.687886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.688588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.688929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.689210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.689965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:46.691903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:46.718163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 873/1742] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 618 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:46.719671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.722035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:46.851289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f2f0e-ff48-4f68-8a8b-89c27274a452 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.853881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:46.865313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.866741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.867428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.867934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.868996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:46.870186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdd4c0-4900-4304-be4f-aeb83fb4590d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:46.879905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:46.882422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:46.883637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:46.884105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:46.884641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:46.901256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 870/1743] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:46.901256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:46.901855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:46.994252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:47.028077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76873e-ad8a-42d6-baa6-4fea3a82479e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.040922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.043994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.045441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.047601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.048081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.049382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.080058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 874/1744] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:47.081088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:47.090470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.097348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.115376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822f73c-70e5-4fd8-8903-1e9581f4521d service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.132453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.133891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.134435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.135286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] PATH_INFO: `/v3/projects/58747613a12c4796905c3b7dbcabc6d7/users/8024bf2d53c34dad8ef59ed5cb0828da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.140745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 871/1745] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:46 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:47.144908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:47.151970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:47.158986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58747613a12c4796905c3b7dbcabc6d7, user_id=8024bf2d53c34dad8ef59ed5cb0828da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.163024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.184477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 875/1746] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:47 2022] PUT /identity/v3/projects/58747613a12c4796905c3b7dbcabc6d7/users/8024bf2d53c34dad8ef59ed5cb0828da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:47.185528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:47.195047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74234e7a-781f-46cb-a702-7b5205204d0c service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.324262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31ba46-4522-4aec-8b8e-1227764d1088 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.339658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.340703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.342237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.342513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] PATH_INFO: `/v3/projects/ac16eee1a511464b8bfc17c09a885306/users/b5b1cdd3671c46bba80d3f106383d54e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.347321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac16eee1a511464b8bfc17c09a885306, user_id=b5b1cdd3671c46bba80d3f106383d54e, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.352878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.382052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.394297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 876/1747] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:47 2022] PUT /identity/v3/projects/ac16eee1a511464b8bfc17c09a885306/users/b5b1cdd3671c46bba80d3f106383d54e/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:47.394297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:47.395085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.395085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.395085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] PATH_INFO: `/v3/projects/1fe6dc98d0824996b0e322503e94327b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.395085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe6dc98d0824996b0e322503e94327b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.397935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.618986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe6dc98d0824996b0e322503e94327b', 'request_id': 'req-22ee5e89-6237-4318-9593-e322bfe4ebb7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:47.644608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe6dc98d0824996b0e322503e94327b', 'request_id': 'req-22ee5e89-6237-4318-9593-e322bfe4ebb7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:47.673714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe6dc98d0824996b0e322503e94327b', 'request_id': 'req-22ee5e89-6237-4318-9593-e322bfe4ebb7'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:47.694078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384d47-970f-45d5-8db1-359c2ef48709 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.720899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] The token cache is being invalidate because project 1fe6dc98d0824996b0e322503e94327b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:47.721447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:47.722230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:47.727523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 872/1748] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:47 2022] DELETE /identity/v3/projects/1fe6dc98d0824996b0e322503e94327b => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:47.731076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:47.735638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:47.754961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:47.916756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.918213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.919152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.921167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.928002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:47.936736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eee481-4be4-4f6d-a02d-5581df25bc9c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:47.946934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee5e89-6237-4318-9593-e322bfe4ebb7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:47.957469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:47.957701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 877/1749] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 570 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:47.957804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.957804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.957804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:47.958030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:47.958030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:47.959667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:47.963360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.034085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] MFA Rules not processed for user `b5b1cdd3671c46bba80d3f106383d54e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:48.036325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.040681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:48.095300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:48.099735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:48.115918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 873/1750] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:48.116958 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:48.119127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.143452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc8a9-e2c8-4125-bd86-73fe7e2f4dc5 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:48.153667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:48.155066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.155525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.155919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] PATH_INFO: `/v3/users/30721e546b21443f9348636e8ee0fa7c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.157738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30721e546b21443f9348636e8ee0fa7c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:48.161222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:48.203328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 878/1751] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:47 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:48.207647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:48.211275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.264161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc113d-0ba1-49dc-b157-09eabf0920b3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:48.266501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30721e546b21443f9348636e8ee0fa7c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.286313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:48.288621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.291107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30721e546b21443f9348636e8ee0fa7c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.292194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.292666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] PATH_INFO: `/v3/users/0fe5a01615e5474486c05dc809dd125d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.296702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe5a01615e5474486c05dc809dd125d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:48.303426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:48.305085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30721e546b21443f9348636e8ee0fa7c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.318156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30721e546b21443f9348636e8ee0fa7c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.319196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30721e546b21443f9348636e8ee0fa7c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.339292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 874/1752] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:48 2022] DELETE /identity/v3/users/30721e546b21443f9348636e8ee0fa7c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:48.340982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:48.345559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.423744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe5a01615e5474486c05dc809dd125d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.458158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe5a01615e5474486c05dc809dd125d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.460846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe5a01615e5474486c05dc809dd125d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.468143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d335e-e372-429f-a574-f83f21385e6b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:48.499545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:48.500669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.501247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.501995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe5a01615e5474486c05dc809dd125d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.502160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] PATH_INFO: `/v3/projects/58747613a12c4796905c3b7dbcabc6d7/users/8024bf2d53c34dad8ef59ed5cb0828da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.508536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58747613a12c4796905c3b7dbcabc6d7, user_id=8024bf2d53c34dad8ef59ed5cb0828da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:48.511661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:48.540227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 875/1753] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:48 2022] PUT /identity/v3/projects/58747613a12c4796905c3b7dbcabc6d7/users/8024bf2d53c34dad8ef59ed5cb0828da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:48.540227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:48.540227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.547495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-53de9c82-cbe8-45f2-8275-c5dc76abbad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe5a01615e5474486c05dc809dd125d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:48.567154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 879/1754] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:48 2022] DELETE /identity/v3/users/0fe5a01615e5474486c05dc809dd125d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:48.568876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbc6b-d6cc-4034-a284-9f0bd4745167 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.571301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbc6b-d6cc-4034-a284-9f0bd4745167 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.572670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbc6b-d6cc-4034-a284-9f0bd4745167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.697529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:48.730276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-25dcbc6b-d6cc-4034-a284-9f0bd4745167 None None] MFA Rules not processed for user `8024bf2d53c34dad8ef59ed5cb0828da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:48.731600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-25dcbc6b-d6cc-4034-a284-9f0bd4745167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:48.742880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:48.799739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24bd4-68cb-4593-be2b-ad16ad97c995 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:48.953814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 880/1755] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:48.954056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.956337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.956337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:48.991613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:48.992235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:48.995947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:48.996221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:48.998561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.000305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.010677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.099236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 881/1756] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:49.100514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:49.101900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.140516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd636af-6836-47a2-9138-4053d9d9a22d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:49.144463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:49.145252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:49.145533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:49.147269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.159668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.274003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:49.280512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:49.315491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 876/1757] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:49.316742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:49.320936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.403415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:49.406874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a2d4f-7935-4c65-859c-02de5368b5f6 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:49.430928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 882/1758] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:49.433196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:49.433535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:49.434021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.461338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:49.462925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.513044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660dc47-8f02-4d86-a214-c3105a9c4f70 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:49.514633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:49.523310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:49.524242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:49.524786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:49.527719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.550804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:49.552603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:49.579685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 883/1760] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:14:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:49.581691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 877/1760] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:49.583229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:49.606849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26f45f-d9d3-4c85-8ef1-40a6f40f04b8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:49.613854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:49.614606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:49.614945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:49.615229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.617607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.785725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:49.789488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:49.828619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 884/1761] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:49.846661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:49.849213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.884182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:49.885633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:49.934767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c103fb1-9dc3-47ad-8090-5a9259ebfa49 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:49.952721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:49.955245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:49.959427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:49.959722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] PATH_INFO: `/v3/projects/fb781a08c1a741c6b84fccf5af550859` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:49.992541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb781a08c1a741c6b84fccf5af550859)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:49.997963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:50.009733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:50.083919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb173390-e034-4ba4-bd1f-5244bc55dbae service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:50.110605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:50.110605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:50.113573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:50.115117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:50.152799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:50.162981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1807c-acb9-465c-b19e-26937da78297 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:50.187347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 878/1762] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:14:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 04:14:50.201889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb781a08c1a741c6b84fccf5af550859', 'request_id': 'req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:50.224579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb781a08c1a741c6b84fccf5af550859', 'request_id': 'req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:50.240644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb781a08c1a741c6b84fccf5af550859', 'request_id': 'req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:50.273134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] The token cache is being invalidate because project fb781a08c1a741c6b84fccf5af550859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:50.273812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:50.274759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50dfa9fe-b31e-4a05-bc3c-355dfbe47157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:50.277886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 885/1763] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:49 2022] DELETE /identity/v3/projects/fb781a08c1a741c6b84fccf5af550859 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:50.283511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:50.283906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:50.284662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:50.345219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:50.350500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:50.496675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 879/1764] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:50.508186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:50.523872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:50.524611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:50.628825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:50.633246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:50.657055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 886/1765] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:50.662327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:50.674594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bee78-a46f-404f-ab57-c79819c7cfb4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:50.696894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:50.701422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:50.701815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:50.702151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:50.703097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:50.703757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:50.705413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:50.773522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 880/1766] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:50.774581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:50.790694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5d720-9c9e-4650-a1b2-de9d78381809 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:50.809678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:50.810441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:50.810680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:50.810956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:50.811690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:50.817226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:50.904536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 887/1767] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:50 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:50.907608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:50.928919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06707f-1fb0-47fd-a076-01cce911eaa0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:51.175887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:51.178962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:51.179439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:51.179752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:51.182528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:51.189732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:51.256968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 881/1768] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:50 2022] POST /identity/v3/users => generated 286 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:51.261439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:51.277310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc16942-3717-4ed8-ad3a-5b440c8da248 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:51.312909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:51.312909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:51.312909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:51.314920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:51.314920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:51.325340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:51.412086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 888/1769] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:51.414125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:51.434903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c19d0-9238-498d-9c46-8b11550300ac admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:51.450537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:51.451247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:51.451534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:51.451814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:51.452950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:51.461113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd9233-b36d-4c20-ad1c-c76c4e4f698c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:51.490584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 882/1770] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:51.494928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:51.513362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5adfa6-d3a0-441a-977e-263357d6815d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:51.527985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:51.528987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:51.529480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:51.530070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] PATH_INFO: `/v3/projects/1288dfde91b340159ac74714c1ac820b/users/b30d48d8025444c59df61ec85069b552/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:51.570913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1288dfde91b340159ac74714c1ac820b, user_id=b30d48d8025444c59df61ec85069b552, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:51.579951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:51.600528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:51.617349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 889/1771] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:51 2022] PUT /identity/v3/projects/1288dfde91b340159ac74714c1ac820b/users/b30d48d8025444c59df61ec85069b552/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:51.633246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:51.633866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:51.634462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:51.696003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] MFA Rules not processed for user `b30d48d8025444c59df61ec85069b552`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:51.700110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:51.794506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:51.834223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:51.836487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:51.926184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 883/1772] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:51.929633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:51.969019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa61d9-dac7-4752-994b-8f0bf92f57f2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:51.980054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:51.981437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.022271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.026999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] PATH_INFO: `/v3/projects/a16deb2e741f484e94c851cddd0cc952` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.032056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16deb2e741f484e94c851cddd0cc952)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.035140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.198646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16deb2e741f484e94c851cddd0cc952', 'request_id': 'req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.219313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f801b-87dc-43f6-8928-278f7e9e4a89 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:52.225317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16deb2e741f484e94c851cddd0cc952', 'request_id': 'req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.265107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16deb2e741f484e94c851cddd0cc952', 'request_id': 'req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.345631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] The token cache is being invalidate because project a16deb2e741f484e94c851cddd0cc952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:52.346392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.352190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8372d6fd-b195-4a9a-9f78-c86f01ab0b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.359168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 890/1773] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:51 2022] DELETE /identity/v3/projects/a16deb2e741f484e94c851cddd0cc952 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:14:52.364610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771bf6-63e8-4c52-bfa9-d72c589ff2e7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.368253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771bf6-63e8-4c52-bfa9-d72c589ff2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.368662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771bf6-63e8-4c52-bfa9-d72c589ff2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.444907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-de771bf6-63e8-4c52-bfa9-d72c589ff2e7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:52.447295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:52.453669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.455023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.455172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-de771bf6-63e8-4c52-bfa9-d72c589ff2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:52.456558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.459357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.463202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.494300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 891/1774] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:52.496474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.496790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.497050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.536896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:52.538972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:52.560171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 892/1775] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:14:52.564413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:52.577363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 884/1776] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:51 2022] POST /identity/v3/projects => generated 371 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:52.577363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:52.582149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:52.591432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4f05b-7708-49e1-82ba-2747278dffd8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:52.702022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:52.702948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.703314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.703654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.705033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:52.705033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.706638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.733968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 893/1777] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:14:52.733968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:52.742850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:52.789760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bdb94-d550-4814-93f2-4449702368a3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:52.813406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:52.814733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.815296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.817211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] PATH_INFO: `/v3/users/72737ccdbc164906a903f58d0ea660df` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.821108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72737ccdbc164906a903f58d0ea660df)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.832675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50e1a-e599-450b-abfc-3861aba28152 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:52.840572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:52.842210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.844964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.845265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.845597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.846430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:52.846828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.848635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.860327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:52.893270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 885/1778] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:52 2022] POST /identity/v3/users => generated 292 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:52.894941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:52.913370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d2e67-21da-4f3d-aa49-efd589ebdcc3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:52.920247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:52.920971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:52.921214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:52.921492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:52.922206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:52.925422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:52.970107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 886/1779] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:52 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:52.970107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:52.974193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72737ccdbc164906a903f58d0ea660df'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.997540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72737ccdbc164906a903f58d0ea660df'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:52.998175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72737ccdbc164906a903f58d0ea660df'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:53.023189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72737ccdbc164906a903f58d0ea660df'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:53.065005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72737ccdbc164906a903f58d0ea660df'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:53.089296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 894/1780] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:52 2022] DELETE /identity/v3/users/72737ccdbc164906a903f58d0ea660df => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:53.090460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.094673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.115031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a54fb-a603-4650-afa3-daf69e17bf2d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.229621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190913b-b705-4e31-963d-95eaa80e6a3f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.236493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.237305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.237597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.237954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.238708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.240781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.244739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.248258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.248685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.249083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.250163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.256466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66461af-9eed-49ad-8cdc-acfaf98a16d7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.276963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 887/1781] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:53.283020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.284253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 895/1782] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:53 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:53.284706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.285665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.285791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.294171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.313790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:53.315588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.322359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4ac1-a3a2-4ded-a33d-e38b104668d3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.329921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 888/1783] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:53.332222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.340219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.341142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.341426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.341706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] PATH_INFO: `/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43/users/d02950ad806c41eeb67d849c787bdca7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.344539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab14e0-df08-4162-b176-0d9b3178eb19 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.351458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.352721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.353152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.353571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.354514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.356465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.362329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff0d0400f884ba4aeff5cd3a617af43, user_id=d02950ad806c41eeb67d849c787bdca7, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.365568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.375172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 889/1784] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:53.381618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.387980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.391451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 896/1785] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:53 2022] PUT /identity/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43/users/d02950ad806c41eeb67d849c787bdca7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:53.396223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.523887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:53.548886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98406c13-cabc-46ea-bfe8-3e6d830f3e29 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.557075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.557906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.558191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.559065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.559873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.564418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.586362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 897/1786] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:53.587454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.605473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11cf30-e2f8-4347-9006-767bd4518b05 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.610436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.611134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.611752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.612165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.613244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.615096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f67e25-b1b9-4803-be45-54ada130672f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.615286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.634958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 898/1787] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:53.645584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:53.648236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.665182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536c30-92a3-4d39-9885-ed7b1ffea654 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.675705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.676630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.676918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.677229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] PATH_INFO: `/v3/projects/2db68dd5ebcc4ed5b6e566c15d576b8b/users/fb6f922089e8479bbb4a9d2092a2d47b/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.695529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db68dd5ebcc4ed5b6e566c15d576b8b, user_id=fb6f922089e8479bbb4a9d2092a2d47b, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.698949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.720895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 899/1788] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:53 2022] PUT /identity/v3/projects/2db68dd5ebcc4ed5b6e566c15d576b8b/users/fb6f922089e8479bbb4a9d2092a2d47b/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:53.726642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.747938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.749601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.749850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.750515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.763234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.765205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e5b53-6ffa-4d30-8d81-d33ab83f1230 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.777956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 890/1789] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:53.779697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.780082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.780433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.814603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] MFA Rules not processed for user `fb6f922089e8479bbb4a9d2092a2d47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:53.817180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.838903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb06c1-3ce7-40ef-934f-26e7c5d27e83 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:53.845159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:53.846473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.847126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.847658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] PATH_INFO: `/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43/users/d02950ad806c41eeb67d849c787bdca7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.850073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff0d0400f884ba4aeff5cd3a617af43, user_id=d02950ad806c41eeb67d849c787bdca7, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:53.853951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6ae18-732d-49ae-9ad5-d48caf39c7ad admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:53.868629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 900/1790] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:53 2022] PUT /identity/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43/users/d02950ad806c41eeb67d849c787bdca7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:53.878710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:53.878988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:53.879272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:53.909885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] MFA Rules not processed for user `d02950ad806c41eeb67d849c787bdca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:53.910500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.951211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 891/1791] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:53.968038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:53.973394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:53.998506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 901/1792] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:54.005506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:54.007081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:54.057474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:54.085219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01702c-51dc-4283-96a5-4703e1a07c5f None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:54.099292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d74eb-3105-43df-84c2-fd0c62e87ff8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:54.105494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:54.106925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:54.107522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:54.107952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:54.121272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:54.123227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:54.140124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 902/1793] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:14:54.158755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:54.160399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:54.161594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:54.161971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:54.178674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:54.194508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:54.201347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:54.305293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:54.307035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:54.324050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 892/1794] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:54.325220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:54.341906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb162-5d03-4622-a6ef-1342ea525bc9 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:54.351476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:54.358417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d5b3-3e95-40ec-9d02-d4bfa36464fb service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:54.358853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:54.359476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:54.359884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:54.362198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:54.365261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:54.366407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:54.366908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:54.367326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] PATH_INFO: `/v3/projects/3aff6e1ced2e4a90b83656f66a4ee05b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:54.368849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aff6e1ced2e4a90b83656f66a4ee05b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:54.371197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:54.489594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aff6e1ced2e4a90b83656f66a4ee05b', 'request_id': 'req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.496905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aff6e1ced2e4a90b83656f66a4ee05b', 'request_id': 'req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.507641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aff6e1ced2e4a90b83656f66a4ee05b', 'request_id': 'req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.514798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:54.516772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:54.524468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] The token cache is being invalidate because project 3aff6e1ced2e4a90b83656f66a4ee05b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:54.524882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.525931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.531038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 903/1795] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:54 2022] DELETE /identity/v3/projects/3aff6e1ced2e4a90b83656f66a4ee05b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:54.532220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:54.534647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:54.547711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 893/1796] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:54.654197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05ca9f-1d6f-4884-ae4e-a2df48ad444f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:54.667444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:54.668669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:54.669056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:54.669435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] PATH_INFO: `/v3/users/226d208d7c8d4e999f245ac9ada88a9d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:54.671844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=226d208d7c8d4e999f245ac9ada88a9d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:54.679838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:54.686719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:54.818128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226d208d7c8d4e999f245ac9ada88a9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.831221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226d208d7c8d4e999f245ac9ada88a9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.833862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226d208d7c8d4e999f245ac9ada88a9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.847323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226d208d7c8d4e999f245ac9ada88a9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.884374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-112453ee-fac7-4630-a2c4-923511aaa41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226d208d7c8d4e999f245ac9ada88a9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:54.907729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 904/1797] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:14:54 2022] DELETE /identity/v3/users/226d208d7c8d4e999f245ac9ada88a9d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:14:55.239050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:55.253925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a49967-423f-4d05-81fb-99bae46fe18f service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:55.326694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:55.327586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.327893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.328156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] PATH_INFO: `/v3/projects/7fc47b0c34044e199788455b83bcb029` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.329382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc47b0c34044e199788455b83bcb029)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:55.332012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:55.386219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc47b0c34044e199788455b83bcb029', 'request_id': 'req-59ffff8b-4284-4192-b8b0-fe51fbff1a41'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:55.393485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc47b0c34044e199788455b83bcb029', 'request_id': 'req-59ffff8b-4284-4192-b8b0-fe51fbff1a41'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:55.396009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:55.400998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc47b0c34044e199788455b83bcb029', 'request_id': 'req-59ffff8b-4284-4192-b8b0-fe51fbff1a41'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:55.418453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] The token cache is being invalidate because project 7fc47b0c34044e199788455b83bcb029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:14:55.418811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:55.419398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-59ffff8b-4284-4192-b8b0-fe51fbff1a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:14:55.421194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 894/1798] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:14:55 2022] DELETE /identity/v3/projects/7fc47b0c34044e199788455b83bcb029 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:55.428385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.428682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.428962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.449707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:55.451037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:55.527399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 905/1799] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:55.528757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.529056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.529342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.553213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:55.554610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:55.565454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 895/1800] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:14:55.570311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:55.582021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1eb4af-902a-4a15-ae03-3a85d9839656 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:55.589257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:55.590960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.590960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.590960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.591914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:14:55.592203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:55.594076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:55.613759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 906/1801] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:14:55.614938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:55.633499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4a65-6293-4063-b255-cd632cd89d36 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:55.654498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:55.655308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.655669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.656011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.656839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:55.658989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:55.717576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 896/1802] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:55 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:55.718627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:55.732948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d765-387f-4076-b32b-1971003df5b6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:55.892284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:55.894898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:55.895309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:55.895667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:55.896664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:55.904723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:55.929957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:55.975619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:55.975755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 907/1803] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:55 2022] POST /identity/v3/users => generated 301 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:55.998188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba357-f353-46ac-8667-4c9651a50a05 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.010162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.011126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.012247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.012542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.020346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.021804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.058469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 897/1804] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:14:56.062997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.079217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218e206-0f3a-4fbb-896d-22199681e1ab admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.097653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.098503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.101285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.101613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.102481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.107994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48271fa-b692-4bfb-86fe-02683d7491b8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.129940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 908/1805] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:56.134717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.156374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ee22b-f8a3-4d36-8925-7bb3e9fbe693 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.164655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.165697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.166072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.166471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] PATH_INFO: `/v3/projects/035fe6cead754a35b0d745e7693ba860/users/d607fb2ac1d64f8d9e7ba873d05ae720/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.195346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035fe6cead754a35b0d745e7693ba860, user_id=d607fb2ac1d64f8d9e7ba873d05ae720, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.198417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.218024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 898/1806] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:56 2022] PUT /identity/v3/projects/035fe6cead754a35b0d745e7693ba860/users/d607fb2ac1d64f8d9e7ba873d05ae720/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:56.235870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.236253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.236543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.272583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] MFA Rules not processed for user `d607fb2ac1d64f8d9e7ba873d05ae720`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:56.273832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:56.354673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 909/1807] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:56.355953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.429519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966deab-d898-488f-ad48-9a8174b8aff8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.434825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.435598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.435953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.436326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.437187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.439266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.472845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 899/1808] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:14:56 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:56.476750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.491735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9979c0-949d-448c-88f2-0f5ef673f404 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.575866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.577140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.577639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.578099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.579076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.581335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.605265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 910/1809] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:14:56 2022] POST /identity/v3/users => generated 306 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:14:56.609477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.621043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169af2-8f2d-40ed-afeb-c4516ad111da admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.627726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.628624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.629014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.629683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.631316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.633383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.649188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 900/1810] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:14:56.653474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.664412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d735e7-d716-4b39-8d27-16e064de853d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.669598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.670657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.671001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.671340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] PATH_INFO: `/v3/projects/079f81f8421841a29bb423f22d4c612d/users/d9bbd96657134b27ba5df02e5b998314/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.705519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=079f81f8421841a29bb423f22d4c612d, user_id=d9bbd96657134b27ba5df02e5b998314, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.709401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.741928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 911/1811] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:56 2022] PUT /identity/v3/projects/079f81f8421841a29bb423f22d4c612d/users/d9bbd96657134b27ba5df02e5b998314/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:56.747513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.758810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a3da-411f-4640-b99b-09b05719aae6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.865314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:56.875907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.879720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.880047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.882633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.883869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.887943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b4a8-2f6e-442d-bcc8-e60ff5a6455a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.903664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 901/1812] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:14:56.908477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:56.921704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55bacc-e464-4a9e-a8ba-9377277e29ff admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:56.927715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:56.928719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.929004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.929280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] PATH_INFO: `/v3/projects/079f81f8421841a29bb423f22d4c612d/users/d9bbd96657134b27ba5df02e5b998314/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.931313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079f81f8421841a29bb423f22d4c612d, user_id=d9bbd96657134b27ba5df02e5b998314, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:56.934122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:56.950611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 912/1813] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:14:56 2022] PUT /identity/v3/projects/079f81f8421841a29bb423f22d4c612d/users/d9bbd96657134b27ba5df02e5b998314/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:14:56.958253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:56.958363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:56.958659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:56.991248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] MFA Rules not processed for user `d9bbd96657134b27ba5df02e5b998314`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:14:56.992548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:57.089546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 902/1814] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:14:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:14:57.099363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:57.101991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:57.154423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:14:57.206872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9603620-88fb-4e08-adee-aad1713a8e41 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:57.211541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:57.212459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:57.212743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:57.213023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:57.229892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:57.241326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:57.265774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 913/1815] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:57.579255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:57.581152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:57.615779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f697c3-9ffb-4c9e-aa97-96e9d3adaf8e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:57.622240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:57.624437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:57.624946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:57.625233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:57.631015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:14:57.713996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:57.715533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0fc4a-080b-4e89-bff2-912f3dbc8155 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:57.735270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 903/1816] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:14:58.019207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:14:58.029667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298559-619f-4c98-beca-3c33538808c7 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:14:58.044009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:14:58.044702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:14:58.046186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:14:58.046510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:14:58.263396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:14:58.266005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:14:58.288196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 914/1817] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:14:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:07.723157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=68 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:08.281823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=42 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:10.377906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:10.378788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:10.378788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:10.406907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:10.408120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:10.463979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 904/1818] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Jul 4 04:15:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:10.514224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:10.516357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:10.519285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:10.520383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:10.532823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261b7df-a23a-438a-acca-e3418196094e service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:10.535880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68343d5-8554-47cc-9cfc-c1e961ca32a7 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:10.537323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:10.538393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:10.538702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:10.544418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:10.544926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:10.545758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:10.546050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:10.546335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:10.552225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:10.553742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:10.554111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:10.555855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:10.562760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 915/1819] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:10.563451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:10.564709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 905/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:10.565667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:10.571838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d589f12-ec38-42b5-b27d-8918e9c8937f service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:10.575118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbab175-d453-4e97-97de-8ab963b509fa service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:10.577313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:10.578799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:10.579477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:10.579765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:10.580122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:10.580250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:10.580560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:10.580921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:10.589771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:10.591422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99510569-7da2-478b-99f4-4d3baec41a44 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:10.596604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:10.598119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270d383-188f-4615-80e5-f14469d07236 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:10.600824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 916/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:10.610815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 906/1822] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:11.450990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:11.451360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:11.451761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:11.453554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 917/1823] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:15:11.460011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb664-3395-40d5-b795-15365f621609 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:11.460562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb664-3395-40d5-b795-15365f621609 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:11.460736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb664-3395-40d5-b795-15365f621609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:11.494747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d5bfb664-3395-40d5-b795-15365f621609 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:11.497314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d5bfb664-3395-40d5-b795-15365f621609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:11.670670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 907/1824] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:15:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:11.690072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:11.692859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:11.714074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31245-c8bd-4a3a-9d75-61af9cde988c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:11.721319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:11.722790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:11.723149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:11.724159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:11.734185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:11.736097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc6237-eb4f-436d-89d3-ddcdb348a9f8 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:11.747331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 918/1825] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:12.737945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:12.738447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:12.738570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:12.740343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 908/1826] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:15:12.757776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:12.758304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:12.758675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:12.800487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:12.801752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:12.815212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 919/1827] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:15:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:12.832294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:12.834512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:12.871507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8af69-f029-409c-9bff-0ddd83f9c912 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:12.883635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:12.885685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:12.886784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:12.892603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:12.906655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:12.911329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bb31e-6a04-4fa3-90d4-3dce678919c6 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:12.926046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 909/1828] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:22.787566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=21 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:22.920588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=72 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:25.166080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:25.262350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37393f9b-a3ec-4708-bdcf-a0d82d3d6f32 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:25.267650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:25.269015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.269575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.270125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] PATH_INFO: `/v3/users/d607fb2ac1d64f8d9e7ba873d05ae720` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.272325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d607fb2ac1d64f8d9e7ba873d05ae720)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:25.274387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:25.354027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd607fb2ac1d64f8d9e7ba873d05ae720'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:25.367840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd607fb2ac1d64f8d9e7ba873d05ae720'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:25.369212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd607fb2ac1d64f8d9e7ba873d05ae720'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:25.380012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd607fb2ac1d64f8d9e7ba873d05ae720'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:25.386349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44125b9-b335-4ba7-b2db-7854ab84a98a None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.386899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44125b9-b335-4ba7-b2db-7854ab84a98a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.386899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44125b9-b335-4ba7-b2db-7854ab84a98a None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.388744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 910/1829] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:15:25.396614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.397115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.397553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.399152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-be88045b-27e9-42bb-a6bb-8d6638a262ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd607fb2ac1d64f8d9e7ba873d05ae720'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:25.407614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 920/1830] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:25 2022] DELETE /identity/v3/users/d607fb2ac1d64f8d9e7ba873d05ae720 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:25.412319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.412614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.412895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.431730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:25.432962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:25.452517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:25.453750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:25.585224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 911/1831] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:25.591469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 921/1832] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:25.602409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:25.608152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:25.668230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:25.669747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:25.702215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868880b-6f67-4dc0-a1b3-ce2e25f0d3f5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:25.709573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:25.710825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.711422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.711890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.722071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5d5b4-b002-4ce5-b31a-e3ebbe13f4d9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:25.724872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:25.726698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:25.727373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:25.727751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:25.727923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:25.729111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:25.738845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:25.740332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:25.746569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 922/1833] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:15:25.751780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 912/1834] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:26.091797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:26.093361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:26.106777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74c2-4014-4633-8276-43d29323069c service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:26.112064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:26.112942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.113223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.113498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] PATH_INFO: `/v3/projects/035fe6cead754a35b0d745e7693ba860` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.114692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035fe6cead754a35b0d745e7693ba860)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:26.116600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:26.118845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:26.174210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035fe6cead754a35b0d745e7693ba860', 'request_id': 'req-82b50f2e-0912-416d-8490-b72504ab0004'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.181365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035fe6cead754a35b0d745e7693ba860', 'request_id': 'req-82b50f2e-0912-416d-8490-b72504ab0004'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.191251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035fe6cead754a35b0d745e7693ba860', 'request_id': 'req-82b50f2e-0912-416d-8490-b72504ab0004'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.207257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] The token cache is being invalidate because project 035fe6cead754a35b0d745e7693ba860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:26.207553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.208213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.209789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 923/1835] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:26 2022] DELETE /identity/v3/projects/035fe6cead754a35b0d745e7693ba860 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:26.213818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:26.223883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331624a-fb1a-412f-b7e7-5dd3892fdd21 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:26.280847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:26.281741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.282025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.282297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] PATH_INFO: `/v3/users/d9bbd96657134b27ba5df02e5b998314` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.283610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bbd96657134b27ba5df02e5b998314)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:26.285539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:26.326235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:26.335576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bbd96657134b27ba5df02e5b998314'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.347259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bbd96657134b27ba5df02e5b998314'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.355975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bbd96657134b27ba5df02e5b998314'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.363794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bbd96657134b27ba5df02e5b998314'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.364497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-31f4cf88-0ad6-4d4d-ad21-f97fe18658d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bbd96657134b27ba5df02e5b998314'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.370690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 913/1836] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:26 2022] DELETE /identity/v3/users/d9bbd96657134b27ba5df02e5b998314 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:26.646640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:26.648294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:26.665451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b50f2e-0912-416d-8490-b72504ab0004 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:26.728039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:26.728843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.729126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.729398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] PATH_INFO: `/v3/projects/079f81f8421841a29bb423f22d4c612d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.730653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] RBAC: Authorizing `identity:delete_project(project_id=079f81f8421841a29bb423f22d4c612d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:26.732566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:26.783988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079f81f8421841a29bb423f22d4c612d', 'request_id': 'req-e8a9919d-fb55-4b83-8ec7-afad123d295a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.791790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079f81f8421841a29bb423f22d4c612d', 'request_id': 'req-e8a9919d-fb55-4b83-8ec7-afad123d295a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.805795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079f81f8421841a29bb423f22d4c612d', 'request_id': 'req-e8a9919d-fb55-4b83-8ec7-afad123d295a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.817676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] The token cache is being invalidate because project 079f81f8421841a29bb423f22d4c612d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:26.818121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.818688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e8a9919d-fb55-4b83-8ec7-afad123d295a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:26.820990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 924/1837] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:26 2022] DELETE /identity/v3/projects/079f81f8421841a29bb423f22d4c612d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:26.825792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.826261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.826359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.857145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:26.866521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:26.921435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 914/1838] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:26.929143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15335db8-acf0-4867-b830-fd5874334802 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.929437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15335db8-acf0-4867-b830-fd5874334802 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.929717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15335db8-acf0-4867-b830-fd5874334802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.947956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-15335db8-acf0-4867-b830-fd5874334802 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:26.949157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-15335db8-acf0-4867-b830-fd5874334802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:26.957498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 925/1839] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:26.961738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:26.971645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2ef35-6cd7-407e-8d86-d6ca5209f67c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:26.976332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:26.977000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:26.977285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:26.977559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:26.978381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:26.978634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:26.980263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:26.996816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 915/1840] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:15:26.998093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15335db8-acf0-4867-b830-fd5874334802 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.007991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15335db8-acf0-4867-b830-fd5874334802 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.012655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.013438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.013712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.014335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.015197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.017094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.027442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:27.047845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 926/1841] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:26 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:27.052115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.064108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81c18-86a6-4911-80a3-433da7c48717 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.129879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.130634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.130925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.131209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.131914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.133878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.158492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 916/1842] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:27 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:27.161262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.172162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb76d6-4867-4789-9752-4b35e94f1832 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.177240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.177906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.178190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.178500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.179238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.180868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.192883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 927/1843] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:27.196963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.207122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d852b-468d-4f51-9788-fdaace4fed8a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.211894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.212544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.212827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.213107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.213837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.215471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898264-65be-414d-a765-a56643f8593b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.226710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 917/1844] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:27.231137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.241700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ae752-b035-4cc8-8099-37e35e0126e0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.247410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.248329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.248709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.249077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] PATH_INFO: `/v3/projects/57869b5f29294acabf4bb271707002cd/users/ac8d7e7584f14a91baa7f3ec3220ef36/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.265572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57869b5f29294acabf4bb271707002cd, user_id=ac8d7e7584f14a91baa7f3ec3220ef36, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.268337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.282398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 928/1845] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:27 2022] PUT /identity/v3/projects/57869b5f29294acabf4bb271707002cd/users/ac8d7e7584f14a91baa7f3ec3220ef36/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:27.293150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.293469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.293763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.321462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] MFA Rules not processed for user `ac8d7e7584f14a91baa7f3ec3220ef36`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:27.322716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:27.331614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:27.394446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 918/1846] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:27.398823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.461839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bee00-bef0-450e-aa17-11216c463ffc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.467002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.467844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.468133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.468412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.469115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.471063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.498880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 929/1847] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:27 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:27.503893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.520917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af6a15-c863-4624-b77c-4d9865a62532 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.586910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.587594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.587882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.588160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.588854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.590813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.611898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 919/1848] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:27 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:27.616576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.628590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ee810-14c3-4a53-81a8-b7f51014360f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.633432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.634130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.634443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.634718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.635459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.637052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.648360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 930/1849] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:27.655562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.671916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae8f8-3b84-455e-930d-f94b9e238a8c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.680392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.681749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.682145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.682457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] PATH_INFO: `/v3/projects/7840214996fc44f0a22307f59920d966/users/8a556079f33242cbb73a89f8bd3abc63/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.700664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7840214996fc44f0a22307f59920d966, user_id=8a556079f33242cbb73a89f8bd3abc63, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.703389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.719394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 920/1850] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:27 2022] PUT /identity/v3/projects/7840214996fc44f0a22307f59920d966/users/8a556079f33242cbb73a89f8bd3abc63/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:27.723952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.734479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e166-4c15-4c0b-82a5-afd5921b3dfd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.806118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.806792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.807226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.807639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.808816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.811464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17b42-4b6f-4257-83d8-63a9c9cc4644 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.822966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 931/1851] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:27.827458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:27.838826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68756d6-624a-4010-8bf4-82524b7e04c2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:27.844678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:27.845587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.845867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.846164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] PATH_INFO: `/v3/projects/7840214996fc44f0a22307f59920d966/users/8a556079f33242cbb73a89f8bd3abc63/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.848344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7840214996fc44f0a22307f59920d966, user_id=8a556079f33242cbb73a89f8bd3abc63, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:27.851131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:27.863126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 921/1852] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:27 2022] PUT /identity/v3/projects/7840214996fc44f0a22307f59920d966/users/8a556079f33242cbb73a89f8bd3abc63/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:27.874435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:27.874732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:27.875040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:27.909450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] MFA Rules not processed for user `8a556079f33242cbb73a89f8bd3abc63`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:27.910801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:27.931631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:27.994755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 932/1853] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:28.006574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:28.008501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:28.070770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e92af-68d4-4459-a62a-dc8dd7e94771 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:28.076215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:28.077501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:28.078068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:28.078343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:28.089793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:28.091186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:28.104670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 922/1854] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:28.203172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:28.214094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9829c-f69d-4161-8e6d-320c41d34068 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:28.218323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:28.219302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:28.219642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:28.219929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:28.283387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:28.284883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca35e06-0e0d-4be5-b3ed-619cffb916eb service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:28.295050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 933/1855] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:28.574775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:28.580820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:28.645926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1041a-2443-4544-a81d-0dba61cdbf45 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:28.651031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:28.651976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:28.652263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:28.652543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] PATH_INFO: `/v3/users/ac8d7e7584f14a91baa7f3ec3220ef36` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:28.653768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8d7e7584f14a91baa7f3ec3220ef36)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:28.655747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:28.704179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8d7e7584f14a91baa7f3ec3220ef36'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:28.716170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8d7e7584f14a91baa7f3ec3220ef36'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:28.726597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8d7e7584f14a91baa7f3ec3220ef36'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:28.734228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8d7e7584f14a91baa7f3ec3220ef36'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:28.735040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8d7e7584f14a91baa7f3ec3220ef36'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:28.741808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 923/1856] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:28 2022] DELETE /identity/v3/users/ac8d7e7584f14a91baa7f3ec3220ef36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:28.746932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:28.747199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:28.747507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:28.765794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:28.767567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:28.835624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 934/1857] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:28.850840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:28.853219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:28.916545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73297-aaac-4b23-89f9-b8bcf796b0d2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:28.920763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:28.921436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:28.921723 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:28.921996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:28.932739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:28.934170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:28.944211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 924/1858] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:29.277780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:29.279344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:29.291870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ee417-34eb-4290-89f6-a90d1a8f866d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:29.296939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:29.297891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:29.298282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:29.298655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] PATH_INFO: `/v3/projects/57869b5f29294acabf4bb271707002cd` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:29.299924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57869b5f29294acabf4bb271707002cd)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:29.302101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:29.349569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:29.352929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57869b5f29294acabf4bb271707002cd', 'request_id': 'req-c5376f08-846b-4108-a5d9-0e9f635ef442'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.360909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57869b5f29294acabf4bb271707002cd', 'request_id': 'req-c5376f08-846b-4108-a5d9-0e9f635ef442'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.370980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57869b5f29294acabf4bb271707002cd', 'request_id': 'req-c5376f08-846b-4108-a5d9-0e9f635ef442'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.385015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] The token cache is being invalidate because project 57869b5f29294acabf4bb271707002cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:29.385463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.386187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.391347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 935/1859] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:29 2022] DELETE /identity/v3/projects/57869b5f29294acabf4bb271707002cd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:29.392889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:29.403568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a654f3-90eb-438a-af7c-aed0555f1ca7 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:29.428880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:29.466858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:29.467767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:29.468056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:29.468337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] PATH_INFO: `/v3/users/8a556079f33242cbb73a89f8bd3abc63` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:29.469562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a556079f33242cbb73a89f8bd3abc63)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:29.471775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:29.524932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a556079f33242cbb73a89f8bd3abc63'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.537377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a556079f33242cbb73a89f8bd3abc63'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.545802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a556079f33242cbb73a89f8bd3abc63'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.552909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a556079f33242cbb73a89f8bd3abc63'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.553551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4e1de27b-ba01-43f4-9fb4-1dcc725d1753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a556079f33242cbb73a89f8bd3abc63'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:29.560721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 925/1860] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:29 2022] DELETE /identity/v3/users/8a556079f33242cbb73a89f8bd3abc63 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:29.923824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:29.926057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:29.939983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5376f08-846b-4108-a5d9-0e9f635ef442 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.000456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.001371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.001824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.002103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] PATH_INFO: `/v3/projects/7840214996fc44f0a22307f59920d966` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.003335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7840214996fc44f0a22307f59920d966)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.005303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.056908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7840214996fc44f0a22307f59920d966', 'request_id': 'req-fe25d41c-e259-49b1-a269-acfc93c0e974'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:30.063537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7840214996fc44f0a22307f59920d966', 'request_id': 'req-fe25d41c-e259-49b1-a269-acfc93c0e974'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:30.072439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7840214996fc44f0a22307f59920d966', 'request_id': 'req-fe25d41c-e259-49b1-a269-acfc93c0e974'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:30.084885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] The token cache is being invalidate because project 7840214996fc44f0a22307f59920d966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:30.085298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:30.086057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fe25d41c-e259-49b1-a269-acfc93c0e974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:30.087895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 936/1861] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:29 2022] DELETE /identity/v3/projects/7840214996fc44f0a22307f59920d966 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:30.093006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.093499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.093604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.113888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:30.115115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:30.304837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 926/1862] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:30.312671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.313167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.313254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.333170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:30.334398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:30.342671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 937/1863] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:30.347081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.356595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3e0a3-e3e9-4a29-ac98-bf2e4b647132 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.361321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.362023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.362305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.362613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.363357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:30.363661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.365253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.381536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 927/1864] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:15:30.382771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.393780 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01424f-1e1e-4811-8e58-9edbe498200e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.398884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.399546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.399831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.400143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.400842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.402770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.434098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 938/1865] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:30 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:30.438696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.449649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cb35-30e5-433f-af62-233758a65043 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.481608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:30.514433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.515128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.515416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.515692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.516419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.518462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.541809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 928/1866] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:30 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:30.546050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.556811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c2ab1-b290-49b7-bd7d-2e2b4c202e0d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.562091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.562803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.563155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.563449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.564301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.565952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.578480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 939/1867] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:30.581444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.592780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00639795-ac37-4312-b795-003e6ebf390b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.598348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.599136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.599535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.599812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.600542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.602143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a654ca0-3601-4a11-962a-e334bce5b19b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.613424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 929/1868] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:30.618095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.628800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3fa76-ce34-4c39-8268-ea3698e7c721 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.635678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.637154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.637154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.637316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] PATH_INFO: `/v3/projects/a27259703f674ebc9a88ed4ba327cf1b/users/7cecd0dfdc084e4dadd6f8924dd44c8a/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.653489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a27259703f674ebc9a88ed4ba327cf1b, user_id=7cecd0dfdc084e4dadd6f8924dd44c8a, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.656200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.666184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:30.674417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 940/1869] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:30 2022] PUT /identity/v3/projects/a27259703f674ebc9a88ed4ba327cf1b/users/7cecd0dfdc084e4dadd6f8924dd44c8a/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:30.682930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.683261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.683547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.710632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] MFA Rules not processed for user `7cecd0dfdc084e4dadd6f8924dd44c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:30.711827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:30.777605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 930/1870] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:30.791340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:30.793381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:30.854388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c9192-1836-4cf9-842a-ecc89bca83ef admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:30.859237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:30.859920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:30.860242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:30.860699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:30.870456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:30.871914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b3dd5-a6a2-4307-9c51-23e084cb8469 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:30.882925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 941/1871] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:32.336574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:32.338949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:32.419568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdbd16-1826-45bc-9148-34bbbe64cc3e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:32.424972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:32.426741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:32.426741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:32.426741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] PATH_INFO: `/v3/users/1da1b96e033d4a84941c8a26752973ce` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:32.428475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da1b96e033d4a84941c8a26752973ce)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:32.430477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:32.536429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:32.539665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da1b96e033d4a84941c8a26752973ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:32.553694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da1b96e033d4a84941c8a26752973ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:32.561806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da1b96e033d4a84941c8a26752973ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:32.571445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da1b96e033d4a84941c8a26752973ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:32.572173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da1b96e033d4a84941c8a26752973ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:32.579038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 931/1872] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:32 2022] DELETE /identity/v3/users/1da1b96e033d4a84941c8a26752973ce => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:32.583872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:32.584332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:32.584446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:32.602558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:32.603805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:32.685398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 942/1873] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:32.696775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:32.698195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:32.757987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c86e-0678-473f-aa2f-f36e0225a2ef admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:32.765319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:32.766557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:32.766981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:32.767356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:32.783021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:32.784755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:32.796208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 932/1874] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:33.272217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:33.275265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:33.295466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0eeee-422b-4620-a337-771a20341a9a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:33.309994 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:33.311005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:33.311293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:33.311572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] PATH_INFO: `/v3/projects/cd464d738d4c451ba7a629970d36917b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:33.312937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd464d738d4c451ba7a629970d36917b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:33.315079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:33.372508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd464d738d4c451ba7a629970d36917b', 'request_id': 'req-d4098ebe-e18c-46e2-a0bc-eeed949d285f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.379609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd464d738d4c451ba7a629970d36917b', 'request_id': 'req-d4098ebe-e18c-46e2-a0bc-eeed949d285f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.389583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd464d738d4c451ba7a629970d36917b', 'request_id': 'req-d4098ebe-e18c-46e2-a0bc-eeed949d285f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.400923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] The token cache is being invalidate because project cd464d738d4c451ba7a629970d36917b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:33.401223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.401876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.403481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 943/1875] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:33 2022] DELETE /identity/v3/projects/cd464d738d4c451ba7a629970d36917b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:33.408310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:33.418584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd466ee-3aa2-4705-936f-a44a2e0ceb29 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:33.479947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:33.480873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:33.481154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:33.481431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] PATH_INFO: `/v3/users/b5b1cdd3671c46bba80d3f106383d54e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:33.482790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b1cdd3671c46bba80d3f106383d54e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:33.484804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:33.572923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b1cdd3671c46bba80d3f106383d54e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.587209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b1cdd3671c46bba80d3f106383d54e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.588955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:33.597002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b1cdd3671c46bba80d3f106383d54e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.606168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b1cdd3671c46bba80d3f106383d54e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.606892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9db55ff8-acd6-4fd7-9776-ee1fbd09f479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b1cdd3671c46bba80d3f106383d54e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:33.618577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 933/1876] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:33 2022] DELETE /identity/v3/users/b5b1cdd3671c46bba80d3f106383d54e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:33.896873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:33.898406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:33.916617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4098ebe-e18c-46e2-a0bc-eeed949d285f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:33.944466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:33.977483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:33.978349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:33.978663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:33.978940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] PATH_INFO: `/v3/projects/ac16eee1a511464b8bfc17c09a885306` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:33.980130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac16eee1a511464b8bfc17c09a885306)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:33.982021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.031779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac16eee1a511464b8bfc17c09a885306', 'request_id': 'req-96ff8548-183b-4d5d-a953-d36f7dc033ef'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:34.038706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac16eee1a511464b8bfc17c09a885306', 'request_id': 'req-96ff8548-183b-4d5d-a953-d36f7dc033ef'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:34.051908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac16eee1a511464b8bfc17c09a885306', 'request_id': 'req-96ff8548-183b-4d5d-a953-d36f7dc033ef'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:34.064169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] The token cache is being invalidate because project ac16eee1a511464b8bfc17c09a885306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:34.064314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:34.065033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ff8548-183b-4d5d-a953-d36f7dc033ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:34.066879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 944/1877] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:33 2022] DELETE /identity/v3/projects/ac16eee1a511464b8bfc17c09a885306 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:34.071955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.072247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.072528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.092277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:34.093591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:34.158891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 934/1878] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:34.166810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.167102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.167384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.185790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:34.187026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:34.195468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 945/1879] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:34.199688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.209730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a29f8-133f-441d-8169-9567b6056225 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.214535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.215205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.215486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.215754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.216529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:34.216838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.218478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.230883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 935/1880] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:34.235474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.244779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690104d0-cbc8-4144-b7dc-63acb354c94b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.249446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.250101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.250393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.250691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.251461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.253359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.284883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 946/1881] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:34.287476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.311901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d1a62-06db-451e-a6ec-0dc264923882 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.377339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.378124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.378541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.378910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.379852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.381855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.407276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 936/1882] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:34.409074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.420551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4c71a-5837-4386-848a-32c07b931267 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.426540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.427255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.427537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.427815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.428560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.430155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.441939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 947/1883] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:34.448055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.459143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74014312-f00a-4690-ba17-e610780082d0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.464905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.465681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.466030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.466393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.467242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.468919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506156cd-0a1b-4d6c-b61a-119c3c2c05e0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.479738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 937/1884] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:34.484068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.494323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963d82-2872-4d3c-b783-bf3b84c2a20c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.498863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.499755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.500040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.500322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] PATH_INFO: `/v3/projects/28c8049ac4b041828fc218c30df01707/users/da54a651bbae493f97f970b13e955425/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.514877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c8049ac4b041828fc218c30df01707, user_id=da54a651bbae493f97f970b13e955425, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.517576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.531977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 948/1885] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:34 2022] PUT /identity/v3/projects/28c8049ac4b041828fc218c30df01707/users/da54a651bbae493f97f970b13e955425/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:34.543735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.544207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.544586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.573792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] MFA Rules not processed for user `da54a651bbae493f97f970b13e955425`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:34.575048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:34.613530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:34.645142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 938/1886] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:34.649787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.702741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:34.714207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b9ea4-1181-450f-b3e8-5e0905990920 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.720126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.720899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.721270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.721598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.722394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.724369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.750083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 949/1887] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:34.754405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.765699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b851a3-f105-40ea-957d-67bced103a88 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.822999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.823595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.823973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.824274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.824982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.826921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.849662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 939/1888] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:34 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:34.853719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.864910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656572-dea9-4236-9e0e-f7012717db53 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.869611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.870326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.870665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.870955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.871697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.873308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.885828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 950/1889] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:34.890790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.902878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053ce78-bc5b-4a2c-9ba3-1a4f579de380 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:34.910711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:34.911633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:34.911970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:34.912251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] PATH_INFO: `/v3/projects/9db9f94440ef4534984beaaec617094f/users/59f76a44314c450a8afe73c31f00b9e0/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:34.928796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db9f94440ef4534984beaaec617094f, user_id=59f76a44314c450a8afe73c31f00b9e0, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:34.931548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:34.950728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 940/1890] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:34 2022] PUT /identity/v3/projects/9db9f94440ef4534984beaaec617094f/users/59f76a44314c450a8afe73c31f00b9e0/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:34.955389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:34.968755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0aa34-d39f-4fab-8917-75a3718adfc2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:35.031671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:35.032361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:35.032733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:35.033017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:35.033757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:35.035412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ac263-9228-4ffb-9609-be6880b02431 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:35.045685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 951/1891] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:35.050104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:35.060648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0eef1-bd76-4418-b038-a73f1e99d7bf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:35.065730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:35.066665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:35.066950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:35.067220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] PATH_INFO: `/v3/projects/9db9f94440ef4534984beaaec617094f/users/59f76a44314c450a8afe73c31f00b9e0/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:35.069318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db9f94440ef4534984beaaec617094f, user_id=59f76a44314c450a8afe73c31f00b9e0, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:35.071994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:35.082385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 941/1892] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:35 2022] PUT /identity/v3/projects/9db9f94440ef4534984beaaec617094f/users/59f76a44314c450a8afe73c31f00b9e0/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:35.092641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:35.092934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:35.093214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:35.123248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] MFA Rules not processed for user `59f76a44314c450a8afe73c31f00b9e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:35.124455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:35.193294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 952/1893] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:35.204265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:35.206231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:35.273336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fb7c-be05-4e10-9b65-d9a10f70ff5d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:35.278926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:35.279862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:35.280277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:35.280709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:35.296520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:35.299468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43dbcc-85e4-4229-95c4-e6524bb546fc service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:35.311478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 942/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:36.783216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:36.843046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:36.859127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053aa5fa-45b0-4413-b53a-647c2df889d3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:36.864159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:36.865055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:36.865373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:36.865680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] PATH_INFO: `/v3/users/da54a651bbae493f97f970b13e955425` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:36.866935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da54a651bbae493f97f970b13e955425)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:36.868846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:36.917761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da54a651bbae493f97f970b13e955425'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:36.928314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da54a651bbae493f97f970b13e955425'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:36.928961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da54a651bbae493f97f970b13e955425'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:36.935900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da54a651bbae493f97f970b13e955425'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:36.948349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da54a651bbae493f97f970b13e955425'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:36.954870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 953/1895] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:36 2022] DELETE /identity/v3/users/da54a651bbae493f97f970b13e955425 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:36.959843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:36.960362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:36.960449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:36.979365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:36.980622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:37.023818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:37.058031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 943/1896] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:37.068275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:37.069691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:37.144291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589686cd-b8ed-42ea-9880-1781cecb406a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:37.150028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:37.151954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:37.154213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:37.155669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:37.166682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:37.168156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:37.188031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 954/1897] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:37.537371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:37.539120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:37.555227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d9a7d-c277-428c-bbc4-e563ed566c5e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:37.560600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:37.561520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:37.561804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:37.562082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] PATH_INFO: `/v3/projects/28c8049ac4b041828fc218c30df01707` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:37.563338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c8049ac4b041828fc218c30df01707)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:37.566910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:37.631138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c8049ac4b041828fc218c30df01707', 'request_id': 'req-89477aa9-035f-4b39-b6a8-22b6323d88c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.638240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c8049ac4b041828fc218c30df01707', 'request_id': 'req-89477aa9-035f-4b39-b6a8-22b6323d88c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.645523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c8049ac4b041828fc218c30df01707', 'request_id': 'req-89477aa9-035f-4b39-b6a8-22b6323d88c9'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.669335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] The token cache is being invalidate because project 28c8049ac4b041828fc218c30df01707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:37.669638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.670420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.672122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 944/1898] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:37 2022] DELETE /identity/v3/projects/28c8049ac4b041828fc218c30df01707 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:37.676515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:37.687779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63d59-4cec-4006-bf96-a7a712031a27 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:37.759406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:37.760557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:37.760841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:37.761314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] PATH_INFO: `/v3/users/59f76a44314c450a8afe73c31f00b9e0` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:37.762992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f76a44314c450a8afe73c31f00b9e0)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:37.766942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:37.782129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:37.828692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f76a44314c450a8afe73c31f00b9e0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.841089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f76a44314c450a8afe73c31f00b9e0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.842512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f76a44314c450a8afe73c31f00b9e0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.856374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f76a44314c450a8afe73c31f00b9e0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.871726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-591ccada-a244-4ae1-9d5f-e4368e36781c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f76a44314c450a8afe73c31f00b9e0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:37.884320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 955/1899] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:37 2022] DELETE /identity/v3/users/59f76a44314c450a8afe73c31f00b9e0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:38.262089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.263662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:38.285505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477aa9-035f-4b39-b6a8-22b6323d88c9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.354255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.355505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.355932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.356370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] PATH_INFO: `/v3/projects/9db9f94440ef4534984beaaec617094f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.357898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db9f94440ef4534984beaaec617094f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.360330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.413730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db9f94440ef4534984beaaec617094f', 'request_id': 'req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:38.420649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db9f94440ef4534984beaaec617094f', 'request_id': 'req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:38.424962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:38.427359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db9f94440ef4534984beaaec617094f', 'request_id': 'req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:38.443477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] The token cache is being invalidate because project 9db9f94440ef4534984beaaec617094f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:38.443780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:38.444557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51d64e2-6a43-4ed1-9563-a0a7f2027abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:38.446545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 945/1900] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:38 2022] DELETE /identity/v3/projects/9db9f94440ef4534984beaaec617094f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:38.452009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.452281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.452558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.472993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:38.474203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:38.557236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 956/1901] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:38.565090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.565382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.565716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.585685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:38.587173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:38.603539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.610501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 946/1902] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:38.619030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5381d-9a0a-4b9e-994e-c86c097a6a19 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.631849 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.633224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.633580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.634007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.635899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:38.636261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.640244 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.658122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 957/1903] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:38.659819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.669649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462a07-9a78-4e3e-ac9b-66946a845c31 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.674732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.675434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.675712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.675989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.676736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.678709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.710618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 947/1904] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:38 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:38.711717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.722165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83564c22-9e6e-4d2c-bd82-c72f891ff806 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.780321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.781009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.781294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.781568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.782288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.784455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.808550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 958/1905] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:38 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:38.810011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.820874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebe252-81a0-4928-8189-88bb4795e29f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.825871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.826639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.826965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.827244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.828114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.829787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.845202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 948/1906] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:38.846317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.858838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c0058-9728-4974-be32-911c4fc47f7e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.863949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.864639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.864914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.865276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.866127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.867822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5f3fb-1f79-45d7-af5b-116b136461c7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.869411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:38.885131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 959/1907] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:38.889563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:38.909271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdd874-9031-49b2-a12d-a155bc899dc5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:38.917874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:38.918986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.919336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.919738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] PATH_INFO: `/v3/projects/68e69fdcb15544d8a1cc4d9afdd8f2b4/users/618fd15a2cd94d63b002109113cfc1b0/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:38.945442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e69fdcb15544d8a1cc4d9afdd8f2b4, user_id=618fd15a2cd94d63b002109113cfc1b0, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:38.948478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:38.965492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 949/1908] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:38 2022] PUT /identity/v3/projects/68e69fdcb15544d8a1cc4d9afdd8f2b4/users/618fd15a2cd94d63b002109113cfc1b0/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:38.976650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:38.976936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:38.977283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.008760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] MFA Rules not processed for user `618fd15a2cd94d63b002109113cfc1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:39.010163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:39.106262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 960/1909] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:39.111023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.194024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515969-ed67-43f6-9c20-ee3db96e8ade admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.199008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.199688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.199976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.200360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.200967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.203274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.239441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 950/1910] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:39 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:39.243544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.260774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895ef0d-eb66-4e31-a491-8852b3aae43a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.352257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.352937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.353421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.353611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.354547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.356729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.378408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 961/1911] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:39 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:39.380632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.392811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4da67-ace2-41de-9dbc-acc786dda5d1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.397537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.398213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.398649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.398910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.399636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.401255 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.413088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 951/1912] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:39.417509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.428471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eaa5ee-4026-412f-a70e-1c4f96c375f0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.433208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.434126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.434438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.434718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] PATH_INFO: `/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3/users/1b8d9e597dd84f1481f4cb92717ad01a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.450342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3486fa409ba74e4b9ddbbb2bf32cc9b3, user_id=1b8d9e597dd84f1481f4cb92717ad01a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.453024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.467804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 962/1913] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:39 2022] PUT /identity/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3/users/1b8d9e597dd84f1481f4cb92717ad01a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:39.474481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.482479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29edd4d-571b-41ee-b604-43b0fc499e72 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.537164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:39.544533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.545146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.545483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.545689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.546504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.549341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e5f9f-985f-4c63-a662-3c9a1904a28a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.560325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 952/1914] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:39.564768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.575373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca051221-8b5a-4e00-8e9b-296b44755897 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.580468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.581362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.581648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.581927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] PATH_INFO: `/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3/users/1b8d9e597dd84f1481f4cb92717ad01a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.584007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3486fa409ba74e4b9ddbbb2bf32cc9b3, user_id=1b8d9e597dd84f1481f4cb92717ad01a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.586661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.597408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 963/1915] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:39 2022] PUT /identity/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3/users/1b8d9e597dd84f1481f4cb92717ad01a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:39.607745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.608040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.608295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.638103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] MFA Rules not processed for user `1b8d9e597dd84f1481f4cb92717ad01a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:39.639685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:39.712487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 953/1916] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:39.723732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.725471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:39.771222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:39.788256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a767613-82ef-47d6-8b2a-cdb0338283cf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.792573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.793228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.793514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.793791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.803990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.805508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a9971-a205-4afd-b652-41d21bf8e07a service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.815386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 964/1917] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:39.848215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:39.917526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbfaf-193d-45de-941e-f7996eed97c2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:39.925110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:39.925997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:39.926275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:39.926552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] PATH_INFO: `/v3/users/618fd15a2cd94d63b002109113cfc1b0` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:39.927903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=618fd15a2cd94d63b002109113cfc1b0)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:39.929950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:39.981091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618fd15a2cd94d63b002109113cfc1b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:39.993752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618fd15a2cd94d63b002109113cfc1b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.002280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618fd15a2cd94d63b002109113cfc1b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.009374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618fd15a2cd94d63b002109113cfc1b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.009998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618fd15a2cd94d63b002109113cfc1b0'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.011455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:40.017001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 954/1918] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:39 2022] DELETE /identity/v3/users/618fd15a2cd94d63b002109113cfc1b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:40.021991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:40.022518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:40.022604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:40.040644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:40.041834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:40.105111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 965/1919] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:40.115139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:40.116672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:40.178535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903928c-356a-458c-8c3c-4059db383eb2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:40.182478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:40.183145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:40.183435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:40.183830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:40.193707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:40.195247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:40.205282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 955/1920] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:15:40.581640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:40.583277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:40.596923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ff7eb-83f1-44c1-b0e6-1e805115de7b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:40.601757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:40.602699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:40.602985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:40.603266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] PATH_INFO: `/v3/projects/68e69fdcb15544d8a1cc4d9afdd8f2b4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:40.604501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e69fdcb15544d8a1cc4d9afdd8f2b4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:40.606489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:40.663151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e69fdcb15544d8a1cc4d9afdd8f2b4', 'request_id': 'req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.670029 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e69fdcb15544d8a1cc4d9afdd8f2b4', 'request_id': 'req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.679535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e69fdcb15544d8a1cc4d9afdd8f2b4', 'request_id': 'req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.691308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] The token cache is being invalidate because project 68e69fdcb15544d8a1cc4d9afdd8f2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:40.691649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.692296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.693946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 966/1921] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:40 2022] DELETE /identity/v3/projects/68e69fdcb15544d8a1cc4d9afdd8f2b4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:40.698166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:40.709750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b2753-4e3d-4134-b768-f572823a8473 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:40.775726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:40.776626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:40.776913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:40.777191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] PATH_INFO: `/v3/users/1b8d9e597dd84f1481f4cb92717ad01a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:40.778403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8d9e597dd84f1481f4cb92717ad01a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:40.780315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:40.829919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8d9e597dd84f1481f4cb92717ad01a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.842761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8d9e597dd84f1481f4cb92717ad01a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.852672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8d9e597dd84f1481f4cb92717ad01a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.859731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8d9e597dd84f1481f4cb92717ad01a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.860369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-99608f0e-11ed-454c-8478-2f1aae1343f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8d9e597dd84f1481f4cb92717ad01a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:40.866590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 956/1922] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:40 2022] DELETE /identity/v3/users/1b8d9e597dd84f1481f4cb92717ad01a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:41.362867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.364303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:41.378623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe1c5-136f-4f74-9bf6-57d601e95ac5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.449437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.450331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.450644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.450921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] PATH_INFO: `/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.452100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3486fa409ba74e4b9ddbbb2bf32cc9b3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.454028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.460718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:41.505272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3486fa409ba74e4b9ddbbb2bf32cc9b3', 'request_id': 'req-7b482e72-4d7c-41c5-8885-ed73493f4c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:41.511534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3486fa409ba74e4b9ddbbb2bf32cc9b3', 'request_id': 'req-7b482e72-4d7c-41c5-8885-ed73493f4c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:41.520269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3486fa409ba74e4b9ddbbb2bf32cc9b3', 'request_id': 'req-7b482e72-4d7c-41c5-8885-ed73493f4c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:41.533197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] The token cache is being invalidate because project 3486fa409ba74e4b9ddbbb2bf32cc9b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:41.533489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:41.534130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7b482e72-4d7c-41c5-8885-ed73493f4c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:41.535944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 967/1923] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:41 2022] DELETE /identity/v3/projects/3486fa409ba74e4b9ddbbb2bf32cc9b3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:41.540644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.540940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.541265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.560295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:41.561507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:41.578056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:41.646482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 957/1924] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:41.655308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.655988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.656340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.675158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:41.676356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:41.684341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 968/1925] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:41.688965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.699205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0491d-7b11-4cd1-8dd0-e32c9a9ce839 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.703868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.704587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.705013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.705291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.706054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:41.706383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.708001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.720764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 958/1926] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:41.724950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.734216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a1d21-fcd9-405f-89fe-e7fd044f6fe0 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.739247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.739904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.740184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.740460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.741127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.743069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.771308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 969/1927] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:41 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:41.775555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.786115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e0c3-a245-45dd-976f-7f7dbaa71ea0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.847566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.848347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.848691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.849031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.849802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.851757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.873703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 959/1928] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:41 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:41.878229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.889520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14841226-33f0-4884-a96f-aa46f2470b09 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.894980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.895654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.895942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.896225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.896978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.898627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.910315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 970/1929] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:41.914458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.924772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49c40-4fc0-47be-a338-681c777e955d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.932270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.933225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.933634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.934009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.935230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.937487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e2ad5-5387-4ff1-963a-8a80d7881a5f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:41.948630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 960/1930] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:41.952803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:41.963241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790cdd4-44bb-4457-bfb1-97241c197be6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:41.968913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:41.969810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:41.970135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:41.970596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] PATH_INFO: `/v3/projects/551d3eeaa97043099ec274b782d41e6a/users/44fa130cdfa94259bf2fcf220181d4f9/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:41.986745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=551d3eeaa97043099ec274b782d41e6a, user_id=44fa130cdfa94259bf2fcf220181d4f9, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:41.989392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.005878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 971/1931] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:41 2022] PUT /identity/v3/projects/551d3eeaa97043099ec274b782d41e6a/users/44fa130cdfa94259bf2fcf220181d4f9/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:42.016866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.017166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.017438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.046212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] MFA Rules not processed for user `44fa130cdfa94259bf2fcf220181d4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:42.047400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:42.114734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 961/1932] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:42.119310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.180280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d65-c1a5-4344-b26f-7b56991b06bd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.188231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.189127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.189463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.189791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.190879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.193862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.227066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 972/1933] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:42 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:42.231335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.242242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6e2dc-07cc-43ae-95b0-bf7eb9db7de2 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.253377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:42.301348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.302009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.302291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.302589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.303290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.305245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.326522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 962/1934] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:42 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:42.330579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.342331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec784e-70e8-426d-b729-96bc7c1f0f20 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.347832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.348508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.348790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.349060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.349797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.351493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.363231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 973/1935] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:42.367557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.378115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75575d-8907-46e4-a5f8-572ea2951cf7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.383814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.384711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.384994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.385271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] PATH_INFO: `/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8/users/c4915b4339154be4b2d895eb72de63ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.401511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7e1cf1199c4ec4803febf1a62f1ae8, user_id=c4915b4339154be4b2d895eb72de63ca, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.404201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.424507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 963/1936] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:42 2022] PUT /identity/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8/users/c4915b4339154be4b2d895eb72de63ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:42.430674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.440107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:42.444298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab6fd-69d7-4408-aa3c-a872acaf98d5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.505500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.506224 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.506507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.506785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.507529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.509140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402002e0-a384-4a04-9ce8-ef1078ea7253 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.521302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 974/1937] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:42.526429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.540063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8a12-6a02-4e31-bcd6-107dc6e61616 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.545623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.546565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.546848 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.547121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] PATH_INFO: `/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8/users/c4915b4339154be4b2d895eb72de63ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.549440 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7e1cf1199c4ec4803febf1a62f1ae8, user_id=c4915b4339154be4b2d895eb72de63ca, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.552231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.563733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 964/1938] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:42 2022] PUT /identity/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8/users/c4915b4339154be4b2d895eb72de63ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:42.573782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.574157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.574549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.601857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] MFA Rules not processed for user `c4915b4339154be4b2d895eb72de63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:42.603171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:42.685263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 975/1939] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:42.696499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.698280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:42.765053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d404-d328-4b4f-b70b-80958053d336 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.769490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.770149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.770458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.770884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.780916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.782405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3110fc-28db-4c70-b0e7-391569ca4a5f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.793565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 965/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:42.861912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:42.937569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dcb69-df10-49a1-94fd-96cdfab6e732 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:42.942354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:42.943272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:42.943554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:42.943857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] PATH_INFO: `/v3/users/44fa130cdfa94259bf2fcf220181d4f9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:42.945085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44fa130cdfa94259bf2fcf220181d4f9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:42.947055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:42.980376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:43.001297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44fa130cdfa94259bf2fcf220181d4f9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.011052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44fa130cdfa94259bf2fcf220181d4f9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.011711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44fa130cdfa94259bf2fcf220181d4f9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.019549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44fa130cdfa94259bf2fcf220181d4f9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.031416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44fa130cdfa94259bf2fcf220181d4f9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.037686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 976/1941] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:42 2022] DELETE /identity/v3/users/44fa130cdfa94259bf2fcf220181d4f9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:43.042636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35188-69fe-4544-9f44-729483466019 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:43.042937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35188-69fe-4544-9f44-729483466019 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:43.043251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35188-69fe-4544-9f44-729483466019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:43.061291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f6d35188-69fe-4544-9f44-729483466019 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:43.062423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f6d35188-69fe-4544-9f44-729483466019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:43.123375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 966/1942] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:43.133324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:43.135090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:43.195258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259e02-1937-49a7-b9c8-6ead88760e06 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:43.198907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:43.199579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:43.199859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:43.200135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:43.209288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:43.210802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:43.220399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 977/1943] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:15:43.509921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d35188-69fe-4544-9f44-729483466019 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:43.511446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f6d35188-69fe-4544-9f44-729483466019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:43.525358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d35188-69fe-4544-9f44-729483466019 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:43.530323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:43.532275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:43.532975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:43.533282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] PATH_INFO: `/v3/projects/551d3eeaa97043099ec274b782d41e6a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:43.534652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=551d3eeaa97043099ec274b782d41e6a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:43.536849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:43.544361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:43.589770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551d3eeaa97043099ec274b782d41e6a', 'request_id': 'req-e989be32-2913-4d91-a4f1-09873e6733bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.596359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551d3eeaa97043099ec274b782d41e6a', 'request_id': 'req-e989be32-2913-4d91-a4f1-09873e6733bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.602336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551d3eeaa97043099ec274b782d41e6a', 'request_id': 'req-e989be32-2913-4d91-a4f1-09873e6733bd'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.616990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] The token cache is being invalidate because project 551d3eeaa97043099ec274b782d41e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:43.617287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.617909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.619645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 967/1944] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:43 2022] DELETE /identity/v3/projects/551d3eeaa97043099ec274b782d41e6a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:43.623746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:43.640909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a9723-3abd-4392-b5e4-1e0d207b3181 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:43.706069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:43.707215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:43.707656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:43.707943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] PATH_INFO: `/v3/users/c4915b4339154be4b2d895eb72de63ca` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:43.709198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4915b4339154be4b2d895eb72de63ca)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:43.711357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:43.780081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4915b4339154be4b2d895eb72de63ca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.782742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:43.790364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4915b4339154be4b2d895eb72de63ca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.791263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4915b4339154be4b2d895eb72de63ca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.801258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4915b4339154be4b2d895eb72de63ca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.818779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-425346d2-5ef9-42d7-9d17-47dd8287845d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4915b4339154be4b2d895eb72de63ca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:43.825398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 978/1945] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:43 2022] DELETE /identity/v3/users/c4915b4339154be4b2d895eb72de63ca => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:44.256258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.257789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:44.272328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989be32-2913-4d91-a4f1-09873e6733bd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.330017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.330953 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.331245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.331523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] PATH_INFO: `/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.332702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7e1cf1199c4ec4803febf1a62f1ae8)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.334660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.385873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7e1cf1199c4ec4803febf1a62f1ae8', 'request_id': 'req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.392750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7e1cf1199c4ec4803febf1a62f1ae8', 'request_id': 'req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.399600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7e1cf1199c4ec4803febf1a62f1ae8', 'request_id': 'req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.419095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] The token cache is being invalidate because project cd7e1cf1199c4ec4803febf1a62f1ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:44.419469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.420149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c5dd8a2d-9185-473f-9917-8e5f2f00e0e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.421918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 968/1946] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:44 2022] DELETE /identity/v3/projects/cd7e1cf1199c4ec4803febf1a62f1ae8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:44.427167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.427542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.428262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.451204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:44.452482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:44.519132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 979/1947] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:44.528305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.528645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.528931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.549947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:44.551240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:44.561938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 969/1948] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:44.566951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.576810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788efb8-e6ba-4be3-b284-86cff896f8d3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.582279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.583061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.583332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.583612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.584355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:44.584667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.586254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.598403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 980/1949] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:44.602834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.612195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec3af2-829d-4841-9f59-771cecdc4c36 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.616817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.617484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.617771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.618046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.618819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.620724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.637854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:44.657721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 970/1950] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:44 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:44.662597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.677346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a23d-dcf3-41d6-a57f-ec108894d6dd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.740508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.741211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.741495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.741773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.742536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.744491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.767133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 981/1951] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:44 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:44.771569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.782265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeca2b-8d18-4402-9921-e6e070047322 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.787311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.788029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.788312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.788592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.789396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.791110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.805204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 971/1952] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:44.806353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.816902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd188-d9aa-4f2c-ae31-0d98b3d0c25d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.821741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.822577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.822866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.823149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.823889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.825517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.837956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 982/1953] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:44.842338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.853976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:44.854775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed946-0540-4e44-a870-c7706c39999a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.855879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:44.861095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.862411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.862783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.862968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] PATH_INFO: `/v3/projects/6f3db55991f34f6d87ea72ccd2add08d/users/009651dbca594dcbb8768dd00ce88f0d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.870566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663037a6-9518-4169-91c7-4f52e1399c48 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:44.876830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:44.878435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.878791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.879216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] PATH_INFO: `/v3/users/7cecd0dfdc084e4dadd6f8924dd44c8a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.881049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cecd0dfdc084e4dadd6f8924dd44c8a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.881743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3db55991f34f6d87ea72ccd2add08d, user_id=009651dbca594dcbb8768dd00ce88f0d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:44.884288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.884603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24370-c2f1-45ca-98be-7af25cf0f197 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:44.909931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 972/1954] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:44 2022] PUT /identity/v3/projects/6f3db55991f34f6d87ea72ccd2add08d/users/009651dbca594dcbb8768dd00ce88f0d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:44.911898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:44.912192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:44.912477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:44.926407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:44.942977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] MFA Rules not processed for user `009651dbca594dcbb8768dd00ce88f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:44.944207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:44.953880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cecd0dfdc084e4dadd6f8924dd44c8a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.967201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cecd0dfdc084e4dadd6f8924dd44c8a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.967983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cecd0dfdc084e4dadd6f8924dd44c8a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.976172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cecd0dfdc084e4dadd6f8924dd44c8a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:44.993635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08016de6-32e5-4349-8198-e269eb84b74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cecd0dfdc084e4dadd6f8924dd44c8a'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.000009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 983/1955] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:44 2022] DELETE /identity/v3/users/7cecd0dfdc084e4dadd6f8924dd44c8a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:45.004534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.004819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.005095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.019506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 973/1956] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:45.024075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.024241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:45.025376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.025480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.106566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 984/1957] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:45.116864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.119239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.167944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619c4b7-8d5b-40b1-821a-1f764faa0080 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.179665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.180816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.181304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.181780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.182750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.187363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.213456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca38d52-eeaa-41ed-ab9e-827951037b2f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.228222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.228931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.229213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.232091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.234436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 974/1958] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:45 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:45.238920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.258618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.261433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.275430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 985/1959] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:45.286758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.287059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:45.290028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.317418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83aefa-0e35-4147-be93-52caeb6f8732 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.318916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd68c-78be-43f0-8bf2-3cf0a8cf8a54 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.323924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.324780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.325068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.325345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.325946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.326458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.327053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.327437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.327789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] PATH_INFO: `/v3/users/fb6f922089e8479bbb4a9d2092a2d47b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.328424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.329731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6f922089e8479bbb4a9d2092a2d47b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.331976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.357307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 975/1960] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:45 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:45.361650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.381181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17852da-c19a-4eca-bd74-7239fbad7202 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.388003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.394782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.395265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.395602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.397144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.403706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.419715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:45.422038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 976/1961] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:45.427129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6f922089e8479bbb4a9d2092a2d47b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.433008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.443434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6f922089e8479bbb4a9d2092a2d47b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.444435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6f922089e8479bbb4a9d2092a2d47b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.448944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43bdb-ef7c-44fd-a9a3-180a5a1d58a3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.463641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6f922089e8479bbb4a9d2092a2d47b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.469049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.469777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.470992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.471278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.472033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.474557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.491527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 977/1962] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:45.498181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.500600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ba435e0d-8125-4d0b-b3fa-119015a0d3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6f922089e8479bbb4a9d2092a2d47b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:45.503055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.517118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 986/1963] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:45 2022] DELETE /identity/v3/users/fb6f922089e8479bbb4a9d2092a2d47b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:45.518846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.519388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.519776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.557888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:45.559919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.566475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad17ee-a729-40d9-8f96-65bc30f68114 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.705767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 987/1964] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:45.716451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.718459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.732545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.733410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.733690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.734007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] PATH_INFO: `/v3/projects/c31cd20ff75e4c32a1eb55ed721d5695/users/7998a0b55be94379901281fb3f5f7835/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.765060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31cd20ff75e4c32a1eb55ed721d5695, user_id=7998a0b55be94379901281fb3f5f7835, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.769628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbc764-46a4-45f0-8aa8-177243f74b1b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:45.785445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 978/1965] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:45 2022] PUT /identity/v3/projects/c31cd20ff75e4c32a1eb55ed721d5695/users/7998a0b55be94379901281fb3f5f7835/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:45.797684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.797911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.798271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.812376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec4557-cd0b-4d5e-aca6-5d0dfbb585ac None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:45.840544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] MFA Rules not processed for user `7998a0b55be94379901281fb3f5f7835`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:45.841865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.889636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:45.891062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:45.891359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:45.891683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:45.942363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:45.949786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 979/1966] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:45.949786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:45.950026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:45.978597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:45.983716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:45.985942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.002220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 988/1967] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:46.003054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.014587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e4c39-6f7e-4966-8acd-6c740ba3cd03 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.020279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.020891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.021228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.021504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.022758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.024911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.070004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d04c8-3166-49e6-8347-5dcb4f7e9e43 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.076527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.078125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.078496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.078862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] PATH_INFO: `/v3/projects/a27259703f674ebc9a88ed4ba327cf1b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.080385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a27259703f674ebc9a88ed4ba327cf1b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.082267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 989/1968] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:46.086449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.087543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.157778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a27259703f674ebc9a88ed4ba327cf1b', 'request_id': 'req-be958c9d-a494-4547-a8f3-6663f6edcfbc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.169524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a27259703f674ebc9a88ed4ba327cf1b', 'request_id': 'req-be958c9d-a494-4547-a8f3-6663f6edcfbc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.182613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a27259703f674ebc9a88ed4ba327cf1b', 'request_id': 'req-be958c9d-a494-4547-a8f3-6663f6edcfbc'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.211781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] The token cache is being invalidate because project a27259703f674ebc9a88ed4ba327cf1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:46.212379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.213902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-be958c9d-a494-4547-a8f3-6663f6edcfbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.218970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 980/1969] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:45 2022] DELETE /identity/v3/projects/a27259703f674ebc9a88ed4ba327cf1b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:46.223894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.224395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.224818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.234154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c9664-8337-49f5-8952-2cedf40cba91 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.264575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:46.268486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.327502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.328879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.329248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.329538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.330262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.333523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b2043-ae2e-4c19-a398-f17dfb09f586 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.357074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 981/1970] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:46.362414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 990/1971] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/users => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:46.364246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.364803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.365236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.365454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.367317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.383256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66522bfd-a9ba-4f6a-ae5d-b9a0277c6ae1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.385375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:46.386697 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.390214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.391021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.391316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.391941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.392486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.394345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.396966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 991/1972] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:46.401628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.410293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 982/1973] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:46.414455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.421699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e243c-6fab-43c0-ae18-28f0f7389540 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.425944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231cd9-0c99-4b80-b3f5-8513e222ad76 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.426798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.427472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.427754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.428035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.428816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:46.429182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.430760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.431959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.432902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.433258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.433549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] PATH_INFO: `/v3/projects/c229faa0e4584262810cbd4d7b3f2f85/users/86c1b87446c54a1c93098899121373ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.444315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 992/1974] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:46.445052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.446796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.455430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c229faa0e4584262810cbd4d7b3f2f85, user_id=86c1b87446c54a1c93098899121373ca, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.459661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.475386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788cfae-e52c-4a2f-b157-d2fb93b30421 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.481415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 983/1975] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:46 2022] PUT /identity/v3/projects/c229faa0e4584262810cbd4d7b3f2f85/users/86c1b87446c54a1c93098899121373ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:46.481415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.498881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:46.500218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:46.546403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.547288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.547572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.548016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] PATH_INFO: `/v3/projects/2db68dd5ebcc4ed5b6e566c15d576b8b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.549438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db68dd5ebcc4ed5b6e566c15d576b8b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.551459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.561850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b6a-feb5-45b6-aaaa-e86b625b9a9e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.567818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.568507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.568799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.569247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.569788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.571729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.605498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 984/1976] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:46.611016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.615357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db68dd5ebcc4ed5b6e566c15d576b8b', 'request_id': 'req-1048f81f-4db4-4318-a026-3fa7aabd7195'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.630063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db68dd5ebcc4ed5b6e566c15d576b8b', 'request_id': 'req-1048f81f-4db4-4318-a026-3fa7aabd7195'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.644441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db68dd5ebcc4ed5b6e566c15d576b8b', 'request_id': 'req-1048f81f-4db4-4318-a026-3fa7aabd7195'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.665175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] The token cache is being invalidate because project 2db68dd5ebcc4ed5b6e566c15d576b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:46.665469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.666138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:46.668155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 993/1977] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:46 2022] DELETE /identity/v3/projects/2db68dd5ebcc4ed5b6e566c15d576b8b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:46.668969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.670269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.703147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a8a37-3780-4fdf-9390-4b504af8d88e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.709572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.710363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.710763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.711136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.712066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.713763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfe95a-10a6-400f-8f52-e766e10d5b8c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.734795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 985/1978] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:46.734795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.734795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.734795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.770200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:46.771363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.787305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048f81f-4db4-4318-a026-3fa7aabd7195 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.789383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 986/1979] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:46.790211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.791595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.794886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.796094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.796520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.797138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.798088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.800672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ce51-4160-46ce-a945-de0f43783b2c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.804872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ee934-3c0b-49af-8aa9-662e42f7e83d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.815891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.816850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.817136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.817474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] PATH_INFO: `/v3/projects/c229faa0e4584262810cbd4d7b3f2f85/users/86c1b87446c54a1c93098899121373ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.825680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c229faa0e4584262810cbd4d7b3f2f85, user_id=86c1b87446c54a1c93098899121373ca, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.828415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.836450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 994/1980] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:46.838079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88ca1a-4782-43a0-a888-36af34287548 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.839161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88ca1a-4782-43a0-a888-36af34287548 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.839515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88ca1a-4782-43a0-a888-36af34287548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.841587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 987/1981] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:46 2022] PUT /identity/v3/projects/c229faa0e4584262810cbd4d7b3f2f85/users/86c1b87446c54a1c93098899121373ca/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:46.842620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.876295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8f88ca1a-4782-43a0-a888-36af34287548 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:46.878248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8f88ca1a-4782-43a0-a888-36af34287548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:46.944226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9044dd-d9cf-429d-af80-e360d6c44443 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.947228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:46.952948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.954079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.954237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.954572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.955563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:46.957653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:46.975912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 995/1982] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:46.978000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 988/1983] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:46.978499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.978948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:46.979191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.979725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.990640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c5eef-fefa-4a6f-ac6f-32929a9bebcf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:46.996534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:46.997260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:46.997549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:46.997859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:46.998664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.000318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.002862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:47.013307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] MFA Rules not processed for user `86c1b87446c54a1c93098899121373ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:47.017895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 989/1984] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:47.018023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:47.019515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.030400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef766fe8-ec4f-4a34-8003-9b5fe5de3562 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.035936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.036598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.036881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.037156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.037892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:47.038201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.039860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.052298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 990/1985] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:47.053082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.062948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f9c7-1860-449b-9aae-c2cec12eb131 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.067926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.068822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.069210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.069491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] PATH_INFO: `/v3/projects/bc11153f9bb04373a78ad73a327f7c8e/users/a5b4d3251cf74134b23263d8934d92f2/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.085570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc11153f9bb04373a78ad73a327f7c8e, user_id=a5b4d3251cf74134b23263d8934d92f2, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.087039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 996/1986] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:47.087840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.088402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.102070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5c1b-624f-493d-9dba-3ce305e3429e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.104239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 991/1987] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:47 2022] PUT /identity/v3/projects/bc11153f9bb04373a78ad73a327f7c8e/users/a5b4d3251cf74134b23263d8934d92f2/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:47.113246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.115133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:47.171956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f959-14a8-44cb-b54b-0ad7a84d5b61 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.175391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.176282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.176471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.176592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.176852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.177159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.177442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.177561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.177749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.179638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08333e95-59ed-4c42-9376-a416bd86039a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.209139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 997/1988] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:47.210870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.211206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.211465 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.240053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] MFA Rules not processed for user `a5b4d3251cf74134b23263d8934d92f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:47.241289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:47.247099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.248546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.258675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 992/1989] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:47.259587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.270105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f69d-9bda-4701-8892-8354e6e0550b service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.286044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:47.321333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 998/1990] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:47.326096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.339431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.340195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.340544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.340840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.341520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.343665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.366310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:47.376815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 993/1991] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:47.385817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.402447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e336f67-065b-4965-b676-b7c3e90ce3fc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.415558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.417327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.417623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.417901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.420420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.426909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.448636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4cac6-6e1f-431e-8a88-047dd5bd3387 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.453203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 994/1992] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:47.458124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.463610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.466072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.466072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.466811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.468721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.474588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.481973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fc70c-fc1a-4a52-be81-24a040f29a3b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.494310 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.495112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.495697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.497817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.499315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.504903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.541468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 995/1993] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:47.546021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.547951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 999/1994] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:47.552581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.561840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2eae8-62e3-4c65-b440-a6797bf4dd51 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.640177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca615d8-3568-41b6-9ecb-12c9a6e4b8aa admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.646274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.647002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.647291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.647658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.648270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.650203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.654075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.655433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.655834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.656850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] PATH_INFO: `/v3/projects/75de4b2884ff49cda822b9d919a25c0d/users/3437d3308c8e4d9d9c629ae242a8fb3c/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.682643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1000/1995] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:47.682862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75de4b2884ff49cda822b9d919a25c0d, user_id=3437d3308c8e4d9d9c629ae242a8fb3c, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.685873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab09fc-325a-4ff9-b771-bbcf5cef9a7a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.688353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.703644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ed586-b8e6-494d-be81-49de2b9e56f7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.706429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 996/1996] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:47 2022] PUT /identity/v3/projects/75de4b2884ff49cda822b9d919a25c0d/users/3437d3308c8e4d9d9c629ae242a8fb3c/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:47.718826 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.719483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.720046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.751839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] MFA Rules not processed for user `3437d3308c8e4d9d9c629ae242a8fb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:47.753129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:47.773753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:47.784504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714978-490c-46e5-b682-99780e221200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.785538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714978-490c-46e5-b682-99780e221200 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.785975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714978-490c-46e5-b682-99780e221200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.786415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95714978-490c-46e5-b682-99780e221200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.787232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714978-490c-46e5-b682-99780e221200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.790117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95714978-490c-46e5-b682-99780e221200 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.815327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1001/1997] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:47.818292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714978-490c-46e5-b682-99780e221200 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.841058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95714978-490c-46e5-b682-99780e221200 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.847160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.858525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.858814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.859094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.862941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.864796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 997/1998] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:47.868250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.869358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.890497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1002/1999] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:47.893339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:47.907849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af33ed-15f0-4ac3-94d3-599231a2d241 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.908269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428bab-ad4d-4b4e-ad99-1f9f00d40ee2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:47.913405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.914270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:47.914530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.914815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.915142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] PATH_INFO: `/v3/projects/543fc14ec0fe44a0a5f3d0a62292d743/users/e0724a735da74ea99db79a3a93b9d4c2/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.915445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.915841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.916187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:47.917146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.920068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.934881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=543fc14ec0fe44a0a5f3d0a62292d743, user_id=e0724a735da74ea99db79a3a93b9d4c2, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:47.938011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494176a5-d908-4ff5-bbf2-eb376fef4f2d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:47.962751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1003/2000] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:47 2022] PUT /identity/v3/projects/543fc14ec0fe44a0a5f3d0a62292d743/users/e0724a735da74ea99db79a3a93b9d4c2/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:47.982723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:47.983013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:47.983292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.018432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 998/2001] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:48.023262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.072113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:48.129534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] MFA Rules not processed for user `e0724a735da74ea99db79a3a93b9d4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:48.135048 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:48.182078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d808-42d6-4595-baf5-86b0437c6d3c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.195767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.196413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.196795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.197326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.198110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.202990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.244954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 999/2002] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:48 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:48.245818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.250207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:48.362247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1004/2003] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:48.362247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.371291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6d2e-368f-4b89-83b1-8f6c46015361 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.381164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.381843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.382128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.382426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.383964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d74624-3d10-404e-a177-12f34fcc3f54 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.385350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:48.392049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.394064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.394608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.395533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.396762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.400301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.413790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1005/2004] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:48.414615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.427612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3cbb-f72d-4c8a-9999-3ebf04aad388 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.433812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.434720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.435156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.435681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.436186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.438288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.468051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.469535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.476120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1006/2005] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:48 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:48.477280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.490510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1000/2006] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:48.491291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.517491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3e48-2d5e-4f46-b9ba-2f898a242943 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.583637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53a51-9e2f-4d4c-97a1-5c664b019581 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.588236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.589145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.589427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.589706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] PATH_INFO: `/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4/users/133dda1bacac4ed1b46c40fa6abd6e77/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.605023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0ed8c3dc7a461eba435e84e4e0d1c4, user_id=133dda1bacac4ed1b46c40fa6abd6e77, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.607665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.609513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.610280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.611106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.611106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.611661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.613903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.617338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:48.630707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1007/2007] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:48 2022] PUT /identity/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4/users/133dda1bacac4ed1b46c40fa6abd6e77/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:48.632183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.634585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:48.645275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1001/2008] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:48 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:48.646169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.651542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:48.658822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf541af2-47db-4ebe-8a47-94cbbc27bedb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.722355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06af39-54f2-4181-ade2-6d2fc4ce9eab admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.727383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.728508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.728786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.729106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] PATH_INFO: `/v3/users/7fececcc55844fe1beb2db7e454342c5` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.730996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fececcc55844fe1beb2db7e454342c5)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.732735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.739937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.742745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.743265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.743773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.745114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.747819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.773484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1002/2009] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:48.774576 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.788704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110f9cc-134c-447b-89fe-c24a192517cc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.794733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.796170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.796290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.796572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.797786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.801215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.820033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fececcc55844fe1beb2db7e454342c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:48.842301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1003/2010] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:48.842301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.850547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fececcc55844fe1beb2db7e454342c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:48.851283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716b06f-72ae-4ef9-9ee1-cc6f8c98cb87 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:48.852285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fececcc55844fe1beb2db7e454342c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:48.858875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:48.860166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.860959 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.861274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] PATH_INFO: `/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4/users/133dda1bacac4ed1b46c40fa6abd6e77/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:48.864546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0ed8c3dc7a461eba435e84e4e0d1c4, user_id=133dda1bacac4ed1b46c40fa6abd6e77, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:48.870705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fececcc55844fe1beb2db7e454342c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:48.874242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:48.901352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1004/2011] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:48 2022] PUT /identity/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4/users/133dda1bacac4ed1b46c40fa6abd6e77/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:48.903968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:48.913484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:48.914949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8d16a7f9-d083-4e24-8f2a-fd436783862e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fececcc55844fe1beb2db7e454342c5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:48.937888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1008/2012] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:48 2022] DELETE /identity/v3/users/7fececcc55844fe1beb2db7e454342c5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:48.939817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2696-9936-411e-970e-cab329f10fab None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:48.941192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2696-9936-411e-970e-cab329f10fab None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:48.941589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2696-9936-411e-970e-cab329f10fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.007845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a4fc2696-9936-411e-970e-cab329f10fab None None] MFA Rules not processed for user `133dda1bacac4ed1b46c40fa6abd6e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:49.009117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a4fc2696-9936-411e-970e-cab329f10fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.033474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf5ac0-8ef6-4182-a7a5-9af70bb92e71 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.110721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.111817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.112200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.112521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] PATH_INFO: `/v3/projects/f17344cf492f44eeb340502781804531/users/729e7696478641e4b3d2a6964cd9cec7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.122156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1009/2013] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:49.123492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.123830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.124437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.132686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:49.136918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17344cf492f44eeb340502781804531, user_id=729e7696478641e4b3d2a6964cd9cec7, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.140708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.154918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:49.157829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.165801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1005/2014] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:48 2022] PUT /identity/v3/projects/f17344cf492f44eeb340502781804531/users/729e7696478641e4b3d2a6964cd9cec7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:49.166533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.172974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.183421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:49.269140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1010/2015] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:49.270082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.285074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db146-749d-4dfc-bb53-34212e8ce57f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.288194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf162f-de09-48a6-b546-adcab5cf106f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.289950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.290628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.290920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.291199 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.293076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.293831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.294645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.294645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.295357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.296950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.315178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1011/2016] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:49.315638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.317352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.374636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.377096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.411263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1006/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:49.412316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.414728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f57d5-7756-461b-98f7-57800d0e64ce admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.420117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.421007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.421221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.421506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.440765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2c3d-d46c-46ff-99ae-e01d13d62906 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.443683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.446253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbc281-0f68-4e2d-9764-b603a22a300a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.464281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.466772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1012/2018] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:49.467037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.467995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.468931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] PATH_INFO: `/v3/projects/f17344cf492f44eeb340502781804531/users/729e7696478641e4b3d2a6964cd9cec7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.475580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17344cf492f44eeb340502781804531, user_id=729e7696478641e4b3d2a6964cd9cec7, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.480029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.497713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1007/2019] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:49 2022] PUT /identity/v3/projects/f17344cf492f44eeb340502781804531/users/729e7696478641e4b3d2a6964cd9cec7/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:49.509068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.509427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.509749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.549149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] MFA Rules not processed for user `729e7696478641e4b3d2a6964cd9cec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:49.550400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.571221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.573381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.636680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1013/2020] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:49.641971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.655890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237eed-90b1-4bdc-8027-530fce18e185 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.660942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.662039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.662724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.662724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] PATH_INFO: `/v3/users/3437d3308c8e4d9d9c629ae242a8fb3c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.664092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3437d3308c8e4d9d9c629ae242a8fb3c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.665459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:49.665964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.712016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0503c-94c1-4481-beba-ca41a3941245 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.717726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.718654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.718997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.719337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.721138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.727983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:49.732779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3437d3308c8e4d9d9c629ae242a8fb3c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:49.751819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3437d3308c8e4d9d9c629ae242a8fb3c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:49.762288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3437d3308c8e4d9d9c629ae242a8fb3c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:49.771618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3437d3308c8e4d9d9c629ae242a8fb3c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:49.772425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c8914bbd-b8e7-4eef-9cc3-790794a0f983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3437d3308c8e4d9d9c629ae242a8fb3c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:49.780592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1008/2021] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:49 2022] DELETE /identity/v3/users/3437d3308c8e4d9d9c629ae242a8fb3c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:49.785258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.785563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.785893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.786671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.788836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.800903 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1014/2022] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:49.804082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:49.805306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.837086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.838947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.858891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd22cc-0384-4218-b535-b06d5a109a33 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.881558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1009/2023] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:49.892060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:49.893692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:49.932890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.934075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.934400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.934713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] PATH_INFO: `/v3/projects/9942552034f44a88a79191475ad6d4d4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.936025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9942552034f44a88a79191475ad6d4d4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:49.938080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:49.970970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fb862-f3bc-4f8d-8726-67db00b26b59 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:49.976903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:49.977744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:49.978716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:49.979049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:49.995156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:50.006795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:50.011448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1010/2024] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:15:50.019578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9942552034f44a88a79191475ad6d4d4', 'request_id': 'req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.028498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9942552034f44a88a79191475ad6d4d4', 'request_id': 'req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.040651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9942552034f44a88a79191475ad6d4d4', 'request_id': 'req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.057038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] The token cache is being invalidate because project 9942552034f44a88a79191475ad6d4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:50.058027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.059722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.061818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1015/2025] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:49 2022] DELETE /identity/v3/projects/9942552034f44a88a79191475ad6d4d4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:50.066335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:50.067895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:50.083799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387294-da64-4444-8edc-6c9ae0a0808d service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:50.187688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:50.190345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:50.190665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:50.190949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] PATH_INFO: `/v3/users/8024bf2d53c34dad8ef59ed5cb0828da` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:50.193851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8024bf2d53c34dad8ef59ed5cb0828da)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:50.199212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:50.210048 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:50.335514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8024bf2d53c34dad8ef59ed5cb0828da'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.359257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8024bf2d53c34dad8ef59ed5cb0828da'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.372827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8024bf2d53c34dad8ef59ed5cb0828da'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.384946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8024bf2d53c34dad8ef59ed5cb0828da'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.385600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8024bf2d53c34dad8ef59ed5cb0828da'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.417428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1011/2026] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:50 2022] DELETE /identity/v3/users/8024bf2d53c34dad8ef59ed5cb0828da => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:50.639826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:50.643756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:50.655310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:50.662187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf1d-8977-49c0-8f58-fa5b85a8c5f2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:50.766593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:50.767514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:50.767922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:50.768268 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] PATH_INFO: `/v3/projects/75de4b2884ff49cda822b9d919a25c0d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:50.769798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75de4b2884ff49cda822b9d919a25c0d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:50.772199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:50.842850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75de4b2884ff49cda822b9d919a25c0d', 'request_id': 'req-88fc1949-f900-455d-917a-cdcb92703554'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.852286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75de4b2884ff49cda822b9d919a25c0d', 'request_id': 'req-88fc1949-f900-455d-917a-cdcb92703554'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.864094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75de4b2884ff49cda822b9d919a25c0d', 'request_id': 'req-88fc1949-f900-455d-917a-cdcb92703554'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.877675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] The token cache is being invalidate because project 75de4b2884ff49cda822b9d919a25c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:50.877964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.878661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:50.880689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1016/2027] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:50 2022] DELETE /identity/v3/projects/75de4b2884ff49cda822b9d919a25c0d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:50.884639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:50.886553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:50.903586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05eb232-f88d-4b1d-85bc-07236360cd76 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:50.916533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:50.918079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:50.989809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:50.991268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:50.991896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:50.992579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] PATH_INFO: `/v3/users/133dda1bacac4ed1b46c40fa6abd6e77` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:50.993891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] RBAC: Authorizing `identity:delete_user(user_id=133dda1bacac4ed1b46c40fa6abd6e77)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:50.996987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.015733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc1949-f900-455d-917a-cdcb92703554 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.021762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.023078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.023500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.023917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] PATH_INFO: `/v3/projects/58747613a12c4796905c3b7dbcabc6d7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.025622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=58747613a12c4796905c3b7dbcabc6d7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:51.027736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.062202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133dda1bacac4ed1b46c40fa6abd6e77'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.076318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:51.079951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133dda1bacac4ed1b46c40fa6abd6e77'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.090494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133dda1bacac4ed1b46c40fa6abd6e77'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.092004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58747613a12c4796905c3b7dbcabc6d7', 'request_id': 'req-96ab921c-e8ac-4d1c-8631-732072723bbb'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.100459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58747613a12c4796905c3b7dbcabc6d7', 'request_id': 'req-96ab921c-e8ac-4d1c-8631-732072723bbb'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.100641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133dda1bacac4ed1b46c40fa6abd6e77'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.101227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-399a4cff-3187-47cd-8429-da9104df994c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133dda1bacac4ed1b46c40fa6abd6e77'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.109702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1012/2028] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:50 2022] DELETE /identity/v3/users/133dda1bacac4ed1b46c40fa6abd6e77 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:51.114576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58747613a12c4796905c3b7dbcabc6d7', 'request_id': 'req-96ab921c-e8ac-4d1c-8631-732072723bbb'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.127440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] The token cache is being invalidate because project 58747613a12c4796905c3b7dbcabc6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:51.127751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.128588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-96ab921c-e8ac-4d1c-8631-732072723bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.130416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1017/2029] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:50 2022] DELETE /identity/v3/projects/58747613a12c4796905c3b7dbcabc6d7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:51.135349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.135644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.135912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.143180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:51.159903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:51.161410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:51.235211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1013/2030] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:51.243410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.243724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.244006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.264389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:51.266001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:51.277416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1018/2031] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:51.282278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.293599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ea921-88f9-4f55-a1cd-9907589b7fa4 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.298900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.299586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.299872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.300168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.301548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:51.301869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:51.303551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.322471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1014/2032] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:51.323755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.339868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83258a6f-c425-4ac1-a7d7-eae5f7dd34e7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.348729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.348729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.348729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.348729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.349501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:51.351657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.416985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1019/2033] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:51 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:51.418997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.431532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c746-8883-4260-946b-89c7a6eea237 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.575178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.575178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.577367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.577685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.578478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:51.584754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.613500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.616246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:51.667488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1015/2034] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:51 2022] POST /identity/v3/users => generated 285 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:51.668558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.670333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:51.689081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b2fde-bc37-4f72-8d4a-0952a1d3797c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.695046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.695961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.696246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.696530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] PATH_INFO: `/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.697883 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0ed8c3dc7a461eba435e84e4e0d1c4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:51.699999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:51.757795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ca5fc-eefb-4040-8bfa-567ab9446980 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:51.783230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0ed8c3dc7a461eba435e84e4e0d1c4', 'request_id': 'req-406c267e-3142-45c7-8699-21e6011645a6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.824448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0ed8c3dc7a461eba435e84e4e0d1c4', 'request_id': 'req-406c267e-3142-45c7-8699-21e6011645a6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.847265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0ed8c3dc7a461eba435e84e4e0d1c4', 'request_id': 'req-406c267e-3142-45c7-8699-21e6011645a6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.871218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] The token cache is being invalidate because project ef0ed8c3dc7a461eba435e84e4e0d1c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:51.871786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.873219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:51.882149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1016/2035] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:51 2022] DELETE /identity/v3/projects/ef0ed8c3dc7a461eba435e84e4e0d1c4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:51.882149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:51.882839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:51.885168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:51.886170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:51.886170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:51.886447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:51.890246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:51.916955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:52.037333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c267e-3142-45c7-8699-21e6011645a6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.054447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.056528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.056872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.057164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.058344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.061554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af682dc-3f87-44a5-b6bf-44dfe4e6c782 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.066979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.069894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.099399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1017/2036] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:52.099399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ea3d-b1b4-4e5d-85df-5cd6276226ed None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.099399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ea3d-b1b4-4e5d-85df-5cd6276226ed None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.099399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4ea3d-b1b4-4e5d-85df-5cd6276226ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.114515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1020/2037] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:52.128468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.158724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb12cd-e127-4d8e-bbdd-13f67b3ebd05 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.168688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-09b4ea3d-b1b4-4e5d-85df-5cd6276226ed None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:52.171248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-09b4ea3d-b1b4-4e5d-85df-5cd6276226ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:52.174837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.175992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.176285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.176730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.177684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.181926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.186221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:52.203628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1018/2038] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:52.205748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.206144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.206822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.226783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1021/2039] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:52.231159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.237726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:52.238907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:52.254465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1019/2040] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:52.255674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.257550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5580a-24a8-408c-9de3-76a8709afbdf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.284784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.287201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.290885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.291198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87bf86-bbbe-4f58-9ed9-b2cb61c3c7eb None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.291317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] PATH_INFO: `/v3/projects/3624a1c31e014c3988edd78e8aa7452c/users/a17ebf9dd6f943a1aab047fb70e85303/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.306344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.307086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.307375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.307697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.310268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:52.310268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.311827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.318240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3624a1c31e014c3988edd78e8aa7452c, user_id=a17ebf9dd6f943a1aab047fb70e85303, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.321255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f5ca-6bec-4d39-81fb-edb26e5a7699 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.343975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1022/2041] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:52 2022] PUT /identity/v3/projects/3624a1c31e014c3988edd78e8aa7452c/users/a17ebf9dd6f943a1aab047fb70e85303/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:52.350703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1020/2042] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:52.358219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.359204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.359921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.360836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.384191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defc50a-a698-4875-8389-73ae22ebea87 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.419359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] MFA Rules not processed for user `a17ebf9dd6f943a1aab047fb70e85303`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:52.421466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:52.512987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.513777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.514448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.514723 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.515432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.517700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.528518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1023/2043] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:52.533572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.552359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abebed1-36a6-4e17-ab39-a3eeb9d0b3f5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.557988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.558956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.559425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.559910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.561498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.562557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1021/2044] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:52.564359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.564855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.607043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1024/2045] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:52.612401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.648643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:52.653370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f810-d4fe-48db-8a55-bb0fabd29bca admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.709706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bf826-272c-4216-afd9-ff4e5208d9b1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.718303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.719095 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.721313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.721670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.722363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.724345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.726259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.726977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.727266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.727547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.728260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.730212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.756415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1025/2046] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:52.763376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.765597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1022/2047] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:52 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:52.770433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.786295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463604-a36a-4b5a-8af6-ff166a2687a3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.793350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.793350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.793598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.793860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.794677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.796787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.806459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115bdad-57c6-49c6-aaa9-3a38fb7c8ad8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.822069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.823817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.824168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1023/2048] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:52.826524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.827194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.827649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.828886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.830750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.841931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe05aea-d707-43c7-b684-2f007f587471 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.849506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.851206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.852930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.853336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.854183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.858136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.862659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1026/2049] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:52.863798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.884058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4affcc-c352-4a38-b1bf-224fc24bc314 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.897227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1024/2050] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:52.898197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.899088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.900014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.900287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.901130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] PATH_INFO: `/v3/projects/8e007c46a77149cd9cb4295635c474cf/users/397a3a613b094c8c8afe8cd30152614a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.915392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8df12-1c40-4607-a02f-ecc95182df69 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:52.930236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:52.931656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:52.932300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:52.932756 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] PATH_INFO: `/v3/projects/1344c2b6e4454674bf798b036c05ca89/users/e2728ec3064b4dfabb48dc3433033511/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:52.937670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e007c46a77149cd9cb4295635c474cf, user_id=397a3a613b094c8c8afe8cd30152614a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.940514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:52.942999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:52.959952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1027/2051] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:52 2022] PUT /identity/v3/projects/8e007c46a77149cd9cb4295635c474cf/users/397a3a613b094c8c8afe8cd30152614a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:52.963782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:52.976027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1344c2b6e4454674bf798b036c05ca89, user_id=e2728ec3064b4dfabb48dc3433033511, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:52.976027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c48de-e3c1-4b44-a643-076d4a316505 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.003784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1025/2052] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:52 2022] PUT /identity/v3/projects/1344c2b6e4454674bf798b036c05ca89/users/e2728ec3064b4dfabb48dc3433033511/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:53.016060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.016388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.016673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.046535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] MFA Rules not processed for user `e2728ec3064b4dfabb48dc3433033511`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:53.047978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.084861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d598c-6286-4bae-a164-a8a66e433310 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.174814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1026/2053] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:53.176079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.178101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.205149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.205868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.206140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.206459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.207239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.209807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.231138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1028/2054] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:53.231906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.235875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:53.247729 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.270066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2db5a-c55c-41e4-a4e3-e52b47a5cd7d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.274079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.274897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.275216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.275577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.280134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.354334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514169df-abc7-434c-b204-d37f9b392594 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.357813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.358503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.358792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.359070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.371561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.373352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.385086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1029/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:53.385531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.393124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.394938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.405317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ba199-c633-4f6e-9429-d1896cce0ac2 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.410139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1027/2056] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:53.414772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.415565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.416147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.416628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.417121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] PATH_INFO: `/v3/projects/8e007c46a77149cd9cb4295635c474cf/users/397a3a613b094c8c8afe8cd30152614a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.421429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e007c46a77149cd9cb4295635c474cf, user_id=397a3a613b094c8c8afe8cd30152614a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.425186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.432140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f3ee8-69ab-4e24-b6f2-d380b65e2f96 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.438222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.438886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.439179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.439562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] PATH_INFO: `/v3/users/e2728ec3064b4dfabb48dc3433033511` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.440851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2728ec3064b4dfabb48dc3433033511)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.444070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.450573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1030/2057] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:53 2022] PUT /identity/v3/projects/8e007c46a77149cd9cb4295635c474cf/users/397a3a613b094c8c8afe8cd30152614a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:53.451388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.489639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:53.531285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2728ec3064b4dfabb48dc3433033511'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:53.539262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4281-e322-4077-9e52-96c2eea4e28c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.544487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.545423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.545772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.546136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.548285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.553137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2728ec3064b4dfabb48dc3433033511'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:53.568971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2728ec3064b4dfabb48dc3433033511'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:53.581232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2728ec3064b4dfabb48dc3433033511'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:53.583493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8b32c6ed-55b5-43b5-9881-eb0fcd327ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2728ec3064b4dfabb48dc3433033511'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:53.594530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1028/2058] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:53 2022] DELETE /identity/v3/users/e2728ec3064b4dfabb48dc3433033511 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:53.599585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.600082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.603697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.634725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.636407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b965-3eb3-4718-9c70-ff32feb6802c service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.660533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1031/2059] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:53.661208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] MFA Rules not processed for user `397a3a613b094c8c8afe8cd30152614a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:53.662108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.662423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.662710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.663365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.696715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:53.698755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.739194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:53.791906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1029/2060] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:53.803795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1032/2061] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:53.815152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.817272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:53.906767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f536c6-9a1e-4026-9cd6-bd56b6a103ba None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:53.911220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:53.912102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:53.913353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:53.913353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:53.926193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:53.928664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:53.945106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1030/2062] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:53.961230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:53.963392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:54.077811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451689a1-b2a8-41fc-8113-a7d3ea9a1338 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.083803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.084723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.085179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.085686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.097569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.099482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.112831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1033/2063] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:54.213178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.222634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9762-c513-4a1b-8297-95f3b0a6949b service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.228011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.228675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.228969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.229248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.231280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:54.274540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.276234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:54.295612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db587b-f26a-4765-a03a-13a775bda829 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.300429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.301330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.301601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.301884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] PATH_INFO: `/v3/projects/1344c2b6e4454674bf798b036c05ca89` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.303132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1344c2b6e4454674bf798b036c05ca89)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.305056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.321579 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.323468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0fbd-203e-4e47-9dd1-ce528e9c6e31 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.338436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1031/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:54.343256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:54.371808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1344c2b6e4454674bf798b036c05ca89', 'request_id': 'req-12d53ace-d832-4903-b295-564246546c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:54.378584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1344c2b6e4454674bf798b036c05ca89', 'request_id': 'req-12d53ace-d832-4903-b295-564246546c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:54.388354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1344c2b6e4454674bf798b036c05ca89', 'request_id': 'req-12d53ace-d832-4903-b295-564246546c09'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:54.399554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] The token cache is being invalidate because project 1344c2b6e4454674bf798b036c05ca89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:54.399843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:54.400472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-12d53ace-d832-4903-b295-564246546c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:54.402306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1034/2065] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:54 2022] DELETE /identity/v3/projects/1344c2b6e4454674bf798b036c05ca89 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:15:54.408034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.408324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.408602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.426135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:54.427374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:54.504245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1032/2066] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:54.511696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.511983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.512318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.532899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:54.534111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:54.542897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1035/2067] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:15:54.548937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.561718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03211-07d4-4e37-91c9-5cf9ffaea055 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.567607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.568492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.569014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.569133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.569979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:54.570362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.572317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.589107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1033/2068] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:54.590260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.600487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2433e5-ee24-4012-b87b-52c29cb84d84 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.610721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.610721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.610721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.610721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.611553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.611553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.662216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1036/2069] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:54 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:54.667906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.680187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc406-98c2-40b9-aee1-2afd800c5838 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.691625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:54.755796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.756493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.756922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.757205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.757925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.759922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.785406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1034/2070] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:54 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:54.786755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.798532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093896-a290-4b66-aabe-9c9c1917a8f2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.803709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.804582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.805109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.805397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.806248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.807941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.820854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1037/2071] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:15:54.825288 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.836651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ddc-dbd9-467e-82ea-01b281ab9d75 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.841368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.842040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.842330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.842691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.843384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.845064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee2220-1f66-4c43-b43e-395af79d3b57 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.855565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1035/2072] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:54.859670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:54.870082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbc21-b198-4758-a9f8-90b929ef3cf2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:54.875107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:54.876037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.876315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.876588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] PATH_INFO: `/v3/projects/7623a8602af14d628c70c1b8b7c84697/users/1fa44ed969c74e9ea476a9cc8284c668/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.893216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7623a8602af14d628c70c1b8b7c84697, user_id=1fa44ed969c74e9ea476a9cc8284c668, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:54.896131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:54.914439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1038/2073] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:54 2022] PUT /identity/v3/projects/7623a8602af14d628c70c1b8b7c84697/users/1fa44ed969c74e9ea476a9cc8284c668/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:54.921753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:54.922025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:54.922325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:54.951670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] MFA Rules not processed for user `1fa44ed969c74e9ea476a9cc8284c668`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:54.952947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:55.020334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1036/2074] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:55.031258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:55.033321 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:55.095630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c48245-f974-4bcf-86d3-21b2384878fc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:55.102265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:55.103047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.103389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.103738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.115405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:55.116825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1735c-b39e-4f88-a3e1-834f715baadf service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:55.125834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1039/2075] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:55.150445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:55.224765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c3a6c-1215-4482-a0e9-53d65536cec6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:55.230393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:55.231351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.231634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.231930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] PATH_INFO: `/v3/users/1fa44ed969c74e9ea476a9cc8284c668` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.233292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa44ed969c74e9ea476a9cc8284c668)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:55.235414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:55.259802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:55.289122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa44ed969c74e9ea476a9cc8284c668'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.301470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa44ed969c74e9ea476a9cc8284c668'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.310983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa44ed969c74e9ea476a9cc8284c668'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.318351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa44ed969c74e9ea476a9cc8284c668'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.319044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa44ed969c74e9ea476a9cc8284c668'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.324964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1037/2076] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:55 2022] DELETE /identity/v3/users/1fa44ed969c74e9ea476a9cc8284c668 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:55.329809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.330088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.330392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.348073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:55.349221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:55.394208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:55.414444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1040/2077] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:55.424265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:55.425765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:55.489251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226fbe2-8642-4319-b5dc-6cf59b6f7c9e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:55.493117 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:55.493768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.494050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.494326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.503547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:55.504988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492732-82d7-401b-9393-909c46a3b35e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:55.514706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1038/2078] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:55.800656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:55.802273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:55.818538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f2bd-1441-43b8-bd28-17146a6f526c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:55.824220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:55.825107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.825388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.825800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] PATH_INFO: `/v3/projects/7623a8602af14d628c70c1b8b7c84697` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.827223 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7623a8602af14d628c70c1b8b7c84697)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:55.829371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:55.884651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7623a8602af14d628c70c1b8b7c84697', 'request_id': 'req-9c7a5179-92bb-43e9-89f1-d5ccf2746267'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.894474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7623a8602af14d628c70c1b8b7c84697', 'request_id': 'req-9c7a5179-92bb-43e9-89f1-d5ccf2746267'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.905225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7623a8602af14d628c70c1b8b7c84697', 'request_id': 'req-9c7a5179-92bb-43e9-89f1-d5ccf2746267'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.920151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] The token cache is being invalidate because project 7623a8602af14d628c70c1b8b7c84697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:55.920576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.921086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9c7a5179-92bb-43e9-89f1-d5ccf2746267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:55.923126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1041/2079] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:55 2022] DELETE /identity/v3/projects/7623a8602af14d628c70c1b8b7c84697 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:55.928716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:55.929244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:55.929322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:55.950729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:55.951588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:56.024319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1039/2080] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:56.032437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.033113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.033113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.052394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:56.053606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:56.061628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1042/2081] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:56.066699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.077471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed3680-c18b-4403-8abb-ffdff7a33669 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.092420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.092420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.092420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.092420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.092717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:15:56.092717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.092717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.105799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1040/2082] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:15:56.110324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.121212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aba305-8755-4af7-9b80-6d96aa401920 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.127843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.128584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.129053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.129219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.129998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.132107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.168638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1043/2083] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:15:56.174614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.187138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65eebe-f98d-4886-86be-5614329b7e86 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.218210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:56.247805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.248476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.248759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.249031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.249724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.251821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.275661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1041/2084] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:15:56 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:15:56.280122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.291764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d3ac3-1719-420a-ba91-fb3a3cdc924f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.302401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.303094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.303375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.303644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.304436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.306047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.319178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1044/2085] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:56.323871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.335803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49197495-e2c7-49de-8286-8ac7ade34661 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.340493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.341151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.341446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.341717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.342460 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.344080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33063bd-babe-42b7-808a-0693ce040f26 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.354800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1042/2086] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:15:56.359058 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.369837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f951e2-f950-4daa-a5d8-46a6c726a740 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.375171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.376096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.376462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.376764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] PATH_INFO: `/v3/projects/c487ee37e54042d680eccb72b046bb55/users/031fb205335f407bb2d275476b98fe24/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.392529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c487ee37e54042d680eccb72b046bb55, user_id=031fb205335f407bb2d275476b98fe24, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.395237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.410932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1045/2087] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:15:56 2022] PUT /identity/v3/projects/c487ee37e54042d680eccb72b046bb55/users/031fb205335f407bb2d275476b98fe24/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:56.417208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.417556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.417929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.437932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:56.439188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:56.498500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1043/2088] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:56.509996 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:56.512322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:56.573502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98173f9-d587-4e3e-95b3-2c10497faa5b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:56.577555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:56.578211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:56.578524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:56.578807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:56.588460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:56.589904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ea8aa-465f-403b-84df-295d8afd1f89 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:56.599784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1046/2089] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:57.978581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:57.981737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:57.998882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040868d-27f9-4c7e-976f-c8f6e3e181f6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:58.005461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:58.006763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:58.007266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:58.007761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] PATH_INFO: `/v3/users/a17ebf9dd6f943a1aab047fb70e85303` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:58.009132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17ebf9dd6f943a1aab047fb70e85303)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:58.011379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:58.093936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17ebf9dd6f943a1aab047fb70e85303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:58.117961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17ebf9dd6f943a1aab047fb70e85303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:58.128068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17ebf9dd6f943a1aab047fb70e85303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:58.145391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17ebf9dd6f943a1aab047fb70e85303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:58.147813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17ebf9dd6f943a1aab047fb70e85303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:58.164155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1044/2090] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:57 2022] DELETE /identity/v3/users/a17ebf9dd6f943a1aab047fb70e85303 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:58.165079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:58.165543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:58.165703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:58.198024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:58.199840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:58.204762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:58.318689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1047/2091] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:15:58.328744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:58.330490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:58.364951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:58.410841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bddcce-50e1-45f6-beb2-f30581fec850 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:58.417972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:58.419045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:58.419705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:58.420155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:58.442348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:58.444121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:58.479587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1045/2092] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:15:58.939371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:58.940921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:58.962436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a2ad2-01e3-4e96-85c4-79d5a082ec23 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:58.967485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:58.968507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:58.968794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:58.969070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] PATH_INFO: `/v3/projects/3624a1c31e014c3988edd78e8aa7452c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:58.970915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3624a1c31e014c3988edd78e8aa7452c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:58.972866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:59.047429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3624a1c31e014c3988edd78e8aa7452c', 'request_id': 'req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.055144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3624a1c31e014c3988edd78e8aa7452c', 'request_id': 'req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.066403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3624a1c31e014c3988edd78e8aa7452c', 'request_id': 'req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.085114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] The token cache is being invalidate because project 3624a1c31e014c3988edd78e8aa7452c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:59.085435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.086238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.088118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1048/2093] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:58 2022] DELETE /identity/v3/projects/3624a1c31e014c3988edd78e8aa7452c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:59.092328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:59.107417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad145-ff4c-400e-81ee-950686327285 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:59.176234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:59.177163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:59.177452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:59.177727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] PATH_INFO: `/v3/users/397a3a613b094c8c8afe8cd30152614a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:59.179030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] RBAC: Authorizing `identity:delete_user(user_id=397a3a613b094c8c8afe8cd30152614a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:59.180970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:59.251096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397a3a613b094c8c8afe8cd30152614a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.267136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397a3a613b094c8c8afe8cd30152614a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.276799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397a3a613b094c8c8afe8cd30152614a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.285363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397a3a613b094c8c8afe8cd30152614a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.286361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-82c21a0c-2528-4be0-8d2a-2fe755b4172c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397a3a613b094c8c8afe8cd30152614a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.293057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1046/2094] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:15:59 2022] DELETE /identity/v3/users/397a3a613b094c8c8afe8cd30152614a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:59.668204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:15:59.669812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:15:59.691963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fe4cb-8cfa-433e-aeca-addfc2c7d7c8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:15:59.796909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:15:59.798125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:59.798913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:59.799302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] PATH_INFO: `/v3/projects/8e007c46a77149cd9cb4295635c474cf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:59.802093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e007c46a77149cd9cb4295635c474cf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:15:59.804233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:15:59.857383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:15:59.871066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e007c46a77149cd9cb4295635c474cf', 'request_id': 'req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.878743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e007c46a77149cd9cb4295635c474cf', 'request_id': 'req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.892227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e007c46a77149cd9cb4295635c474cf', 'request_id': 'req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.924017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] The token cache is being invalidate because project 8e007c46a77149cd9cb4295635c474cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:15:59.924342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.925009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ed82cf88-6aab-4b5d-8376-313d8c9e26b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:15:59.926635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1049/2095] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:15:59 2022] DELETE /identity/v3/projects/8e007c46a77149cd9cb4295635c474cf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:15:59.931552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:15:59.931845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:15:59.932126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:15:59.964642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:15:59.966133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:00.049663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:00.076184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1047/2096] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:00.083851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.084269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.084560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.105612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:00.106990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:00.116282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1050/2097] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:00.120389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.131091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648f15-c078-4ab8-b09d-0a9298c5760a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.136052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.136837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.137782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.137782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.139332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:00.140043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.141447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.155438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1048/2098] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:00.159626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.171485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6127ca-bffc-4acd-8dda-803bdf35e0cd None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.177228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.177980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.178785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.178785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.179529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.181495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.213930 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1051/2099] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/projects => generated 407 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:00.218251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.229501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd973194-7b2b-43b8-ab45-405ec0603cec admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.309695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.310587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.310975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.311351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.312216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.314602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.339990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1049/2100] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/users => generated 304 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:00.343988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.355686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4459c-e35d-4c54-b929-a98a4ba3ece7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.360709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.361496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.361880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.362259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.363135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.364851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.377548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1052/2101] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:00.381952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.395076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b11b-1ba6-4438-a0ae-9a4468367a6e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.400411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.401093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.401620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.401907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.403041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.405375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada214af-d12a-4552-8543-ce299a3d2e11 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.417462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1050/2102] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:00.422074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.435439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a1980-d357-400d-a0d6-73a7057045de admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.442408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.443757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.444201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.444759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] PATH_INFO: `/v3/projects/a47bc20cd3a14cb2beeb494e1d14910c/users/fa7f31e8f5f84c7a8be9528073094daf/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.467445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47bc20cd3a14cb2beeb494e1d14910c, user_id=fa7f31e8f5f84c7a8be9528073094daf, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.471259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.490448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1053/2103] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:00 2022] PUT /identity/v3/projects/a47bc20cd3a14cb2beeb494e1d14910c/users/fa7f31e8f5f84c7a8be9528073094daf/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:00.499689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.499988 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.500270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.540417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] MFA Rules not processed for user `fa7f31e8f5f84c7a8be9528073094daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:00.541811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:00.669730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1051/2104] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:00.670977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.795548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271003b-8e43-416a-b4c1-669aead5fa3b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.805951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.806759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.807049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.807378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.808114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.812286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:00.860133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1054/2105] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/projects => generated 409 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:00.863948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:00.885230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea58af-c7de-4f72-9a8f-7ef5ae1badda None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:00.940527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:00.970049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:00.970782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:00.971084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:00.971543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:00.972249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:00.974989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.000220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1052/2106] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:00 2022] POST /identity/v3/users => generated 311 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:16:01.004880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.016580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb16f9d-ab27-42e8-8a69-18c9a5bae61a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.022758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.023618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.023899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.024178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.024928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.026933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.039893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1055/2107] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:01.051594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.062646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7c628-0f68-4f4c-bbda-1e14823e34a5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.078559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.079900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.080189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.080844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] PATH_INFO: `/v3/projects/95e2e57ec8124fb4bdbb1be05579c951/users/bb256acf2ae24bc0abf2d1ea974af275/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.105632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e2e57ec8124fb4bdbb1be05579c951, user_id=bb256acf2ae24bc0abf2d1ea974af275, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.114950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.138027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1053/2108] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:01 2022] PUT /identity/v3/projects/95e2e57ec8124fb4bdbb1be05579c951/users/bb256acf2ae24bc0abf2d1ea974af275/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:01.138789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.157319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82571e-5c8c-4003-8d55-c2a7d0813001 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.204369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:01.250197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.253030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.253497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.254581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.255479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.257286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63f136-5103-402a-9abd-14b2e9746f4f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.271214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1056/2109] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:01.275787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.291121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdb42-fb53-4935-bc2b-029617e4971e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.296886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.298011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.298404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.298792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] PATH_INFO: `/v3/projects/95e2e57ec8124fb4bdbb1be05579c951/users/bb256acf2ae24bc0abf2d1ea974af275/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.301068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e2e57ec8124fb4bdbb1be05579c951, user_id=bb256acf2ae24bc0abf2d1ea974af275, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.304019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf4bbe-f07c-4985-bd44-0dedd803839a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.317140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1054/2110] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:01 2022] PUT /identity/v3/projects/95e2e57ec8124fb4bdbb1be05579c951/users/bb256acf2ae24bc0abf2d1ea974af275/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:01.329918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.329918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.330358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.381405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] MFA Rules not processed for user `bb256acf2ae24bc0abf2d1ea974af275`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:01.382794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:01.425314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.425607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.425886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.483504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] MFA Rules not processed for user `031fb205335f407bb2d275476b98fe24`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:01.485951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:01.487827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1057/2111] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:01.501049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.505019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:01.587980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15a863-a3b9-4510-b9a4-1308aa7a7826 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.591611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1055/2112] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:01.591779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.592460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.592747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.593082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.596499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.597828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:01.604006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.605463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.616188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1058/2113] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:01.675026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b3216-9677-42df-8177-9402b79b4e06 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.680673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.681394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.681680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.681955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.682712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.684701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.729741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1056/2114] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:01.731107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.742891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0397e4-f9be-4f7f-a6c2-40c1c46c89b3 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.815049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:01.817301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.818285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.818789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.819376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.820377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.821854 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.822737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.832957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:01.859529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1059/2115] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:01 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:01.861451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.873475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59409e7a-eadd-4c27-ae2f-8c0757524e77 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.881753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.882585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.882872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.883209 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.884232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.885681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.892868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0553f64-2555-4d7d-898c-a2254171d027 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.900163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1060/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:01.900367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.900367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.900367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.900367 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.901687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.914441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14105edb-74bd-4ad9-b673-a4cc9a8c746a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:01.918075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:01.919125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:01.919478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:01.919823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] PATH_INFO: `/v3/projects/30f6ed3c34b04720ba0587134fa1b300/users/fd20974ac5b44d9c9ea3e5bad4d3189d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:01.938255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f6ed3c34b04720ba0587134fa1b300, user_id=fd20974ac5b44d9c9ea3e5bad4d3189d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.942277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:01.962072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1061/2117] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:01 2022] PUT /identity/v3/projects/30f6ed3c34b04720ba0587134fa1b300/users/fd20974ac5b44d9c9ea3e5bad4d3189d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:01.966243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:01.989633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:01.991604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:02.005384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1057/2118] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:02.048457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9b93-2035-4553-a213-cd894c42de12 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:02.053505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:02.054262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:02.054609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:02.054871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:02.055619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:02.057235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:02.069153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1062/2119] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:02.072888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:02.084975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb686235-2113-4782-9695-7ff7c40eb009 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:02.090231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:02.091380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:02.091672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:02.091955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] PATH_INFO: `/v3/projects/30f6ed3c34b04720ba0587134fa1b300/users/fd20974ac5b44d9c9ea3e5bad4d3189d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:02.094111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f6ed3c34b04720ba0587134fa1b300, user_id=fd20974ac5b44d9c9ea3e5bad4d3189d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:02.097549 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d24fb5-93b1-41cd-8897-815fb703b53b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:02.113201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1058/2120] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:02 2022] PUT /identity/v3/projects/30f6ed3c34b04720ba0587134fa1b300/users/fd20974ac5b44d9c9ea3e5bad4d3189d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:05.815361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:05.817128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:05.849712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87667b-b515-4dcf-8ffa-edb252af3645 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:05.952925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:05.954432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:05.954864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:05.955290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] PATH_INFO: `/v3/users/009651dbca594dcbb8768dd00ce88f0d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:05.956745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009651dbca594dcbb8768dd00ce88f0d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:05.959469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:06.062847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009651dbca594dcbb8768dd00ce88f0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:06.066945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:06.077240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009651dbca594dcbb8768dd00ce88f0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:06.079018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009651dbca594dcbb8768dd00ce88f0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:06.097011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009651dbca594dcbb8768dd00ce88f0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:06.118876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009651dbca594dcbb8768dd00ce88f0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:06.144077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1063/2121] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:05 2022] DELETE /identity/v3/users/009651dbca594dcbb8768dd00ce88f0d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:06.148726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:06.149239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:06.149239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:06.200425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:06.216563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:06.384282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1059/2122] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:06.397510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:06.401608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:06.551925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067b70-7b3e-4126-898e-b44242430883 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:06.560427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:06.561133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:06.561412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:06.562834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:06.595078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:06.604567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:06.639901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1064/2123] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:07.294704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:07.296276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:07.317103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43de53-b286-470c-b96d-081b51d50157 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:07.331593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:07.344297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:07.345031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:07.346282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] PATH_INFO: `/v3/projects/6f3db55991f34f6d87ea72ccd2add08d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:07.351232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3db55991f34f6d87ea72ccd2add08d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:07.370296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:07.520222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:07.528529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3db55991f34f6d87ea72ccd2add08d', 'request_id': 'req-5e3a82b4-7842-49ca-9272-7f2748c85816'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.552002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3db55991f34f6d87ea72ccd2add08d', 'request_id': 'req-5e3a82b4-7842-49ca-9272-7f2748c85816'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.567235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3db55991f34f6d87ea72ccd2add08d', 'request_id': 'req-5e3a82b4-7842-49ca-9272-7f2748c85816'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.601663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] The token cache is being invalidate because project 6f3db55991f34f6d87ea72ccd2add08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:07.601970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.605047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.608684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1060/2124] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:07 2022] DELETE /identity/v3/projects/6f3db55991f34f6d87ea72ccd2add08d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:07.613001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:07.638300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707ea1c-1be0-4239-935d-ecc663ce486d service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:07.760872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:07.761835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:07.762221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:07.762487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] PATH_INFO: `/v3/users/7998a0b55be94379901281fb3f5f7835` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:07.766652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7998a0b55be94379901281fb3f5f7835)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:07.775442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:07.869250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7998a0b55be94379901281fb3f5f7835'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.892384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7998a0b55be94379901281fb3f5f7835'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.893019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7998a0b55be94379901281fb3f5f7835'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.905583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7998a0b55be94379901281fb3f5f7835'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.916332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:07.923799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a6e95e18-b7c7-4b30-8729-000f92c11d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7998a0b55be94379901281fb3f5f7835'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:07.936556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1065/2125] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:07 2022] DELETE /identity/v3/users/7998a0b55be94379901281fb3f5f7835 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:08.426498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:08.437387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:08.438041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:08.438597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:08.439158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:08.463444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a82b4-7842-49ca-9272-7f2748c85816 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:08.489405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] MFA Rules not processed for user `fd20974ac5b44d9c9ea3e5bad4d3189d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:08.490721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:08.561340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:08.562438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:08.562819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:08.563029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] PATH_INFO: `/v3/projects/c31cd20ff75e4c32a1eb55ed721d5695` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:08.564330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31cd20ff75e4c32a1eb55ed721d5695)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:08.566588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:08.640484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31cd20ff75e4c32a1eb55ed721d5695', 'request_id': 'req-719d6904-8e3f-49a3-8d9e-da6ed854eff2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.648236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31cd20ff75e4c32a1eb55ed721d5695', 'request_id': 'req-719d6904-8e3f-49a3-8d9e-da6ed854eff2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.663229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31cd20ff75e4c32a1eb55ed721d5695', 'request_id': 'req-719d6904-8e3f-49a3-8d9e-da6ed854eff2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.671062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1066/2126] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:08.686211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:08.687782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:08.691972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] The token cache is being invalidate because project c31cd20ff75e4c32a1eb55ed721d5695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:08.692280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.692943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.697942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1061/2127] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:08 2022] DELETE /identity/v3/projects/c31cd20ff75e4c32a1eb55ed721d5695 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:08.698940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:08.700325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:08.766917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d9674-3947-4641-adc9-dece15bcae4c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:08.778360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d6904-8e3f-49a3-8d9e-da6ed854eff2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:08.784800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:08.786086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:08.786405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:08.786680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] PATH_INFO: `/v3/users/86c1b87446c54a1c93098899121373ca` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:08.787914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c1b87446c54a1c93098899121373ca)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:08.790379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:08.793267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:08.839831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:08.840662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:08.840952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:08.841239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:08.843150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:08.873302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c1b87446c54a1c93098899121373ca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.889904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c1b87446c54a1c93098899121373ca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.908383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c1b87446c54a1c93098899121373ca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.921401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:08.930138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c1b87446c54a1c93098899121373ca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.932450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a53f114f-41e8-4e31-b7ab-a613c67a7c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c1b87446c54a1c93098899121373ca'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:08.940828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:08.943490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:08.949320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1062/2128] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:08 2022] DELETE /identity/v3/users/86c1b87446c54a1c93098899121373ca => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:08.960062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1067/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:09.153086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.153334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.153597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.197608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:09.198872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:09.279707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1063/2130] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:09.324678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:09.326276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:09.401469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:09.402930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:09.412972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcba1d-a623-4129-80f8-8dba32da7a32 service glance] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:09.477967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a368e23-60da-4cca-afd3-1b62b366b289 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:09.483256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:09.484446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.484733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.485305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] PATH_INFO: `/v3/projects/c229faa0e4584262810cbd4d7b3f2f85` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.486583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c229faa0e4584262810cbd4d7b3f2f85)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:09.488182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:09.488533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:09.488847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.489129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.489431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.499614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:09.501880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e3d63-c688-41df-a235-4b5cb9d945be service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:09.513009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1068/2131] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:16:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:16:09.526590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:09.560896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c229faa0e4584262810cbd4d7b3f2f85', 'request_id': 'req-f98cbb46-e92d-4293-889a-280f11c2da4a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:09.571856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c229faa0e4584262810cbd4d7b3f2f85', 'request_id': 'req-f98cbb46-e92d-4293-889a-280f11c2da4a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:09.582077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c229faa0e4584262810cbd4d7b3f2f85', 'request_id': 'req-f98cbb46-e92d-4293-889a-280f11c2da4a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:09.621091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] The token cache is being invalidate because project c229faa0e4584262810cbd4d7b3f2f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:09.621397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:09.622093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f98cbb46-e92d-4293-889a-280f11c2da4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:09.625929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1064/2132] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:09 2022] DELETE /identity/v3/projects/c229faa0e4584262810cbd4d7b3f2f85 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:09.632557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.633088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.634519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.676264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:09.678272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:09.768722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1069/2133] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:09.776637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.776908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.777187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.807554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:09.811537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:09.837271 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1065/2134] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:09.841239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:09.852899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba88bb-4b79-4c8e-a449-fd23f4772a68 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:09.859586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:09.861356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.861843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.862252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.865222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:09.865222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:09.865222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:09.892295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1070/2135] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:09.894747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:09.912691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60481e05-c232-43f0-808d-08e33cd9fc9c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:09.920572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:09.921437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:09.921809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:09.922813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:09.923645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:09.926209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:09.963124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1066/2136] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:09 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:09.967404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:09.976141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:09.984587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40927d-4c06-4b39-9025-8232f764e37e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.073915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.074979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.075430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.076456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.077598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.080503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:10.152086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.153082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1071/2137] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:09 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:16:10.185597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fefaf1-0f0f-4093-ab7c-32e4d5af1323 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.208921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.222979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.223638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.224115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.225283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.227724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:10.249441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1067/2138] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:10.254073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.272835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2141f-37ba-4200-9812-feac6f0dfdc1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.280374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.281242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.282453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.284800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.284800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.287689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30386d9-e0d5-4661-9fa7-439d6152d0c5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:10.305314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1072/2139] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:10.308653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.324894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c229-8ca8-4d94-b907-f9981d39210d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.339580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.344686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.345488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.346001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] PATH_INFO: `/v3/projects/eab14214d5d74fd9b9258e5e1d0fc6f5/users/9fefd8be3731448392057333ba9eb076/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.370936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab14214d5d74fd9b9258e5e1d0fc6f5, user_id=9fefd8be3731448392057333ba9eb076, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.374507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:10.398015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1068/2140] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:10 2022] PUT /identity/v3/projects/eab14214d5d74fd9b9258e5e1d0fc6f5/users/9fefd8be3731448392057333ba9eb076/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:10.410906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.411315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.411673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.516096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] MFA Rules not processed for user `9fefd8be3731448392057333ba9eb076`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:10.518694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:10.759813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1073/2141] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:10.764686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.805502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.807287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:10.882857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadf111-e6f0-4828-822d-71a2e35e49f4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.901922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.903126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.906628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.908075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.909047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.914710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:10.961067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d5f5-1aad-4427-b346-7d1caa396a56 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:10.971475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1069/2142] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:10 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:10.973872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:10.975838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:10.977041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:10.978070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:10.979592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] PATH_INFO: `/v3/users/a5b4d3251cf74134b23263d8934d92f2` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:10.981762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b4d3251cf74134b23263d8934d92f2)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:10.985790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.028043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:11.078674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:11.098216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b4d3251cf74134b23263d8934d92f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:11.112230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b4d3251cf74134b23263d8934d92f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:11.113144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b4d3251cf74134b23263d8934d92f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:11.123701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b4d3251cf74134b23263d8934d92f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:11.124319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d5047-d75a-4a79-9cf0-3df58390e5bf admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.130871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.131628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.131918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.132304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.133145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.135281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.138061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6e57c59b-69ae-4c12-84cc-c9ca8cfa8a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b4d3251cf74134b23263d8934d92f2'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:11.165885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1074/2143] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:10 2022] DELETE /identity/v3/users/a5b4d3251cf74134b23263d8934d92f2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:11.170280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.172119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.173788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.181152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1070/2144] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:10 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:11.184863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:11.189254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:11.210704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7765eb-2cac-49d1-9c08-67275d5c2c85 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.242305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:11.248585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:11.336787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.337969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.338479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.338863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.339876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.342111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.356794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1075/2145] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:11.364847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:11.366813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:11.386272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1071/2146] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:16:11.408302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:11.443575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5440-430e-4f2b-8e8b-9f22d51983d9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.451100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.452933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.453515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.454274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] PATH_INFO: `/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc/users/9b0b49b90dfc482f9d995f0ed4292874/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.457160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8c81d-0c40-46f4-b1ce-54fcbb201c35 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.466999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.468447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.468737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.469020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.484377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fc22a9d07e43eb88e1b408f58d18bc, user_id=9b0b49b90dfc482f9d995f0ed4292874, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.488363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.493015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.496735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.510423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1076/2147] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:16:11.529392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1072/2148] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:11 2022] PUT /identity/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc/users/9b0b49b90dfc482f9d995f0ed4292874/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:11.531499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:11.543730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328c23-6460-4bf6-9f38-29c85e460891 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.590115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:11.693853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.693853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.693853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.693853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.694393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.698566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010217-a44d-493e-8191-12506d030313 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.731341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1077/2149] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:11.735159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:11.758183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48317872-5687-4716-981d-83a6333b5e5f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:11.775564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:11.787721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.788023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.788315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] PATH_INFO: `/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc/users/9b0b49b90dfc482f9d995f0ed4292874/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:11.793627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fc22a9d07e43eb88e1b408f58d18bc, user_id=9b0b49b90dfc482f9d995f0ed4292874, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:11.797453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:11.865525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1073/2150] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:11 2022] PUT /identity/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc/users/9b0b49b90dfc482f9d995f0ed4292874/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:11.875836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:11.876590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:11.876736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:12.001013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] MFA Rules not processed for user `9b0b49b90dfc482f9d995f0ed4292874`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:12.002938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:12.107394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1078/2151] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:12.119624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:12.121817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:12.163008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:12.164776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:12.244526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f497a28-a012-4638-8c12-7a1c287dba39 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:12.256751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:12.257534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:12.257989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:12.258431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:12.274770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:12.276350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:12.291687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1074/2152] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:12.297370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b6943-d7b4-4a39-9245-700fad3695c1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:12.308187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:12.309096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:12.312442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:12.312805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] PATH_INFO: `/v3/projects/bc11153f9bb04373a78ad73a327f7c8e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:12.316276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc11153f9bb04373a78ad73a327f7c8e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:12.321875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:12.461944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc11153f9bb04373a78ad73a327f7c8e', 'request_id': 'req-b2497a7f-2fef-4fd6-afb9-23467943cb16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.473717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc11153f9bb04373a78ad73a327f7c8e', 'request_id': 'req-b2497a7f-2fef-4fd6-afb9-23467943cb16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.479612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:12.489212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc11153f9bb04373a78ad73a327f7c8e', 'request_id': 'req-b2497a7f-2fef-4fd6-afb9-23467943cb16'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.541583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] The token cache is being invalidate because project bc11153f9bb04373a78ad73a327f7c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:12.541975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.544495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.546970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1079/2153] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:12 2022] DELETE /identity/v3/projects/bc11153f9bb04373a78ad73a327f7c8e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:12.551061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:12.573808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5da5b7-96f8-4de9-ad48-2db9244a4ed0 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:12.696480 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:12.775600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:12.780320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:12.781019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:12.782792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] PATH_INFO: `/v3/users/e0724a735da74ea99db79a3a93b9d4c2` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:12.787139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0724a735da74ea99db79a3a93b9d4c2)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:12.792173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:12.899970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0724a735da74ea99db79a3a93b9d4c2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.948041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0724a735da74ea99db79a3a93b9d4c2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:12.971816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0724a735da74ea99db79a3a93b9d4c2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.003300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0724a735da74ea99db79a3a93b9d4c2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.005342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-379c05ad-b038-4880-afd1-2c8a40f65b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0724a735da74ea99db79a3a93b9d4c2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.023552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1075/2154] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:12 2022] DELETE /identity/v3/users/e0724a735da74ea99db79a3a93b9d4c2 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:13.059935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:13.061526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:13.085327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2497a7f-2fef-4fd6-afb9-23467943cb16 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:13.232504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:13.233881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:13.234387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:13.234968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] PATH_INFO: `/v3/users/fa7f31e8f5f84c7a8be9528073094daf` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:13.238719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7f31e8f5f84c7a8be9528073094daf)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:13.242338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:13.418875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7f31e8f5f84c7a8be9528073094daf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.438586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7f31e8f5f84c7a8be9528073094daf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.440357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7f31e8f5f84c7a8be9528073094daf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.451444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7f31e8f5f84c7a8be9528073094daf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.493852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7f31e8f5f84c7a8be9528073094daf'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.512769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1080/2155] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:13 2022] DELETE /identity/v3/users/fa7f31e8f5f84c7a8be9528073094daf => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:13.515149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:13.516905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:13.517570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:13.566866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:13.571437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:13.775547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1076/2156] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:13.811598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:13.813479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:13.814629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:13.818746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:13.853570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a4e42-d4da-44a3-bf4b-30138d5e64b1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:13.862275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:13.863373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:13.863764 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:13.864144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] PATH_INFO: `/v3/users/9fefd8be3731448392057333ba9eb076` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:13.865931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fefd8be3731448392057333ba9eb076)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:13.868800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:13.947706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e46e4-eb6d-478c-8f1a-85c468c2736e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:13.952045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:13.952770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:13.953056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:13.953339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:13.953999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:13.959257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fefd8be3731448392057333ba9eb076'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:13.966985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:13.968498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:13.981824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1081/2157] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:13.982829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:13.984768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:13.987662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fefd8be3731448392057333ba9eb076'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.000255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b0e6-a5ac-423d-a187-1646daaeb72f service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:14.002006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fefd8be3731448392057333ba9eb076'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.005614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:14.006650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:14.006942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:14.007227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] PATH_INFO: `/v3/projects/543fc14ec0fe44a0a5f3d0a62292d743` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:14.008988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] RBAC: Authorizing `identity:delete_project(project_id=543fc14ec0fe44a0a5f3d0a62292d743)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:14.011975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:14.012598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fefd8be3731448392057333ba9eb076'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.013337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e8e31508-ad7e-439b-8b10-076a28b14a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fefd8be3731448392057333ba9eb076'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.016011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:14.031563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1077/2158] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:13 2022] DELETE /identity/v3/users/9fefd8be3731448392057333ba9eb076 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:14.033840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:14.035067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:14.037680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:14.075769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:14.077337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:14.136143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543fc14ec0fe44a0a5f3d0a62292d743', 'request_id': 'req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.152789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543fc14ec0fe44a0a5f3d0a62292d743', 'request_id': 'req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.165126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543fc14ec0fe44a0a5f3d0a62292d743', 'request_id': 'req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.169170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1078/2159] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:14.176994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:14.178913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:14.181014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] The token cache is being invalidate because project 543fc14ec0fe44a0a5f3d0a62292d743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:14.181878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.183281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.185635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1082/2160] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:13 2022] DELETE /identity/v3/projects/543fc14ec0fe44a0a5f3d0a62292d743 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:14.191181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:14.193674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:14.279796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaa86b-9600-4917-95d4-6b188a733248 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:14.318059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3ab7-e79b-44af-ae7b-ecd65693340d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:14.332746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:14.334009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:14.334698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:14.335046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] PATH_INFO: `/v3/users/729e7696478641e4b3d2a6964cd9cec7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:14.337476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=729e7696478641e4b3d2a6964cd9cec7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:14.339621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:14.394976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:14.395670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:14.395956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:14.396233 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:14.399024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:14.417679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:14.420038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:14.447253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1079/2161] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:14.455184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729e7696478641e4b3d2a6964cd9cec7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.472453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729e7696478641e4b3d2a6964cd9cec7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.473192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729e7696478641e4b3d2a6964cd9cec7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.489584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729e7696478641e4b3d2a6964cd9cec7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.505363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729e7696478641e4b3d2a6964cd9cec7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:14.512685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1083/2162] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:14 2022] DELETE /identity/v3/users/729e7696478641e4b3d2a6964cd9cec7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:14.780275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:14.781918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:14.808182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec738b6f-41f3-4301-afe6-14b83558f272 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:14.874513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:15.052840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:15.054079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.058820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.060313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] PATH_INFO: `/v3/projects/a47bc20cd3a14cb2beeb494e1d14910c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:15.065179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47bc20cd3a14cb2beeb494e1d14910c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:15.074725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:15.171073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47bc20cd3a14cb2beeb494e1d14910c', 'request_id': 'req-e0316388-b0e3-40ca-b92e-1243ea6d8794'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.180464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47bc20cd3a14cb2beeb494e1d14910c', 'request_id': 'req-e0316388-b0e3-40ca-b92e-1243ea6d8794'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.205191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47bc20cd3a14cb2beeb494e1d14910c', 'request_id': 'req-e0316388-b0e3-40ca-b92e-1243ea6d8794'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.273507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] The token cache is being invalidate because project a47bc20cd3a14cb2beeb494e1d14910c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:15.274223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.275634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.278512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1080/2163] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:14 2022] DELETE /identity/v3/projects/a47bc20cd3a14cb2beeb494e1d14910c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:15.283306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:15.284201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:15.318030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7dd98-c91c-436b-92a7-c09fcba26cc4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:15.326029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:15.328197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:15.341565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:15.448985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:15.450184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.450682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.450947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] PATH_INFO: `/v3/users/bb256acf2ae24bc0abf2d1ea974af275` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:15.453640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb256acf2ae24bc0abf2d1ea974af275)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:15.456560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:15.479410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316388-b0e3-40ca-b92e-1243ea6d8794 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:15.495457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:15.496762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.498302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.498922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] PATH_INFO: `/v3/projects/eab14214d5d74fd9b9258e5e1d0fc6f5` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:15.500590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab14214d5d74fd9b9258e5e1d0fc6f5)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:15.503573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:15.583072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb256acf2ae24bc0abf2d1ea974af275'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.607347 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb256acf2ae24bc0abf2d1ea974af275'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.608942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb256acf2ae24bc0abf2d1ea974af275'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.620777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb256acf2ae24bc0abf2d1ea974af275'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.666613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb256acf2ae24bc0abf2d1ea974af275'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.692059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1084/2164] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:15 2022] DELETE /identity/v3/users/bb256acf2ae24bc0abf2d1ea974af275 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:15.698580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:15.700348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:15.708659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab14214d5d74fd9b9258e5e1d0fc6f5', 'request_id': 'req-213f8830-c8f7-4640-a9e3-2bb94fca3e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.712843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:15.721292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab14214d5d74fd9b9258e5e1d0fc6f5', 'request_id': 'req-213f8830-c8f7-4640-a9e3-2bb94fca3e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.743229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab14214d5d74fd9b9258e5e1d0fc6f5', 'request_id': 'req-213f8830-c8f7-4640-a9e3-2bb94fca3e97'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.769957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] The token cache is being invalidate because project eab14214d5d74fd9b9258e5e1d0fc6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:15.770258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.770987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.775711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1081/2165] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:15 2022] DELETE /identity/v3/projects/eab14214d5d74fd9b9258e5e1d0fc6f5 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:15.777195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:15.780750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:15.833250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2cd3d-8ec7-492b-b98e-a5f33cd59f48 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:15.839002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:15.840324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.840617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.840900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] PATH_INFO: `/v3/projects/f17344cf492f44eeb340502781804531` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:15.842433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17344cf492f44eeb340502781804531)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:15.845561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:15.891860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8830-c8f7-4640-a9e3-2bb94fca3e97 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:15.899176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:15.899432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:15.900777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.901065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.901483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] PATH_INFO: `/v3/users/9b0b49b90dfc482f9d995f0ed4292874` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:15.904657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0b49b90dfc482f9d995f0ed4292874)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:15.907293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:15.915801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17344cf492f44eeb340502781804531', 'request_id': 'req-028bd04a-3f7a-430d-9b44-c72e18fd7776'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.928156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17344cf492f44eeb340502781804531', 'request_id': 'req-028bd04a-3f7a-430d-9b44-c72e18fd7776'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.945792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17344cf492f44eeb340502781804531', 'request_id': 'req-028bd04a-3f7a-430d-9b44-c72e18fd7776'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.980487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] The token cache is being invalidate because project f17344cf492f44eeb340502781804531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:15.980487 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.981113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-028bd04a-3f7a-430d-9b44-c72e18fd7776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:15.983022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1085/2166] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:15 2022] DELETE /identity/v3/projects/f17344cf492f44eeb340502781804531 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:15.988799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f56e4a-a6d5-4799-8760-0fe935f839fb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:15.989264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f56e4a-a6d5-4799-8760-0fe935f839fb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:15.989622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f56e4a-a6d5-4799-8760-0fe935f839fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.001240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0b49b90dfc482f9d995f0ed4292874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.016391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c8f56e4a-a6d5-4799-8760-0fe935f839fb None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:16.018631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c8f56e4a-a6d5-4799-8760-0fe935f839fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.024867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0b49b90dfc482f9d995f0ed4292874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.039492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0b49b90dfc482f9d995f0ed4292874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.052051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0b49b90dfc482f9d995f0ed4292874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.053401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0b49b90dfc482f9d995f0ed4292874'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.062074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1082/2167] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:15 2022] DELETE /identity/v3/users/9b0b49b90dfc482f9d995f0ed4292874 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:16.131177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.134477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.182636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1086/2168] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:16.186541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.186794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.187021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.213182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:16.214479 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.285276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0817cb-dd49-4cea-bc4e-5591f257aec5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:16.291463 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:16.295380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.297120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.298179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] PATH_INFO: `/v3/projects/95e2e57ec8124fb4bdbb1be05579c951` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.305272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e2e57ec8124fb4bdbb1be05579c951)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:16.311909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:16.335409 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:16.374664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1087/2169] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:16.380826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.410208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768182-03de-4c4c-9924-d33feb170a6b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:16.423293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:16.424003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.424294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.424577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.426475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:16.427107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:16.429803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:16.450698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e2e57ec8124fb4bdbb1be05579c951', 'request_id': 'req-656869f0-a918-4b60-8eba-0fb70c1836d8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.459722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e2e57ec8124fb4bdbb1be05579c951', 'request_id': 'req-656869f0-a918-4b60-8eba-0fb70c1836d8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.466241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1088/2170] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:16.472631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.491702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e12fa-c820-4b6b-9edc-4f1ab4b8f1a6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:16.492069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e2e57ec8124fb4bdbb1be05579c951', 'request_id': 'req-656869f0-a918-4b60-8eba-0fb70c1836d8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.532893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] The token cache is being invalidate because project 95e2e57ec8124fb4bdbb1be05579c951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:16.534380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.535968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-656869f0-a918-4b60-8eba-0fb70c1836d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.541202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1083/2171] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:16 2022] DELETE /identity/v3/projects/95e2e57ec8124fb4bdbb1be05579c951 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:16.546095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.546403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.546887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.571658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:16.576277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:16.583578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.592457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:16.593200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.593494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.593874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.595160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:16.606973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:16.660536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1089/2172] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:16.662097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.664928 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.710207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1084/2173] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:16.710207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.710207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.721306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339296-dfca-453d-9b90-fc2d71ea7d73 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:16.765878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c23f-1cdc-48a9-997d-3a6a31e2f631 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:16.772444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:16.773338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.773621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.773898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] PATH_INFO: `/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.775687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fc22a9d07e43eb88e1b408f58d18bc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:16.778015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:16.805192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:16.805888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.806323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.806649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.807564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:16.809516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd1db9-1498-4cc8-9b29-deee90ea5685 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:16.834338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1085/2174] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:16.835862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d77952-bd0f-4779-8501-6da68cb7ffa6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.836155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d77952-bd0f-4779-8501-6da68cb7ffa6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.836430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d77952-bd0f-4779-8501-6da68cb7ffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.859500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fc22a9d07e43eb88e1b408f58d18bc', 'request_id': 'req-777cb062-58a4-4e07-9fa3-74f7cc88bc60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.861442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b6d77952-bd0f-4779-8501-6da68cb7ffa6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:16.862690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b6d77952-bd0f-4779-8501-6da68cb7ffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.870587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fc22a9d07e43eb88e1b408f58d18bc', 'request_id': 'req-777cb062-58a4-4e07-9fa3-74f7cc88bc60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.880040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:16.886411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fc22a9d07e43eb88e1b408f58d18bc', 'request_id': 'req-777cb062-58a4-4e07-9fa3-74f7cc88bc60'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.918292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] The token cache is being invalidate because project 29fc22a9d07e43eb88e1b408f58d18bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:16.918658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.919579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:16.926522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1090/2175] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:16 2022] DELETE /identity/v3/projects/29fc22a9d07e43eb88e1b408f58d18bc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:16.926522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:16.930201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.953214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1086/2176] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:16.953214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:16.953214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:16.953214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:16.974267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:16.975799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:16.985471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1087/2177] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:16.986547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.004062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87bbf0-c968-4b15-a191-421fb12d2be5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.009190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.009892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.010204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.010530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.011283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:17.011573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.013218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972decdf-13f3-49af-affc-151b1bdb6032 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.027932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1088/2178] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:17.033099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.033392 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.033678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.047843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cb062-58a4-4e07-9fa3-74f7cc88bc60 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.054447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:17.054983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.055689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:17.055904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.056142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.056415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.057167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.059425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.064862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1089/2179] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:17.065812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.067800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:17.076259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1091/2180] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.077751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.079239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdd85b-290d-48fa-bdf4-82be0e938f79 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.085757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.086585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.086941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.087169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.088020 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.089952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.092842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33893a5-0e60-4ff1-a393-74c76eae5a1a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.098821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.099656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.100006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.100346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.101150 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:17.101521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.103247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.122549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1092/2181] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:17.126287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.137203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1090/2182] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:17.138624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.139504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0a84-f35a-4bc1-8eb5-4ff3845691bb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.225057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a44948-8f3c-489c-9830-8da7e72a4719 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.232344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.233348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.233824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.234317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.235513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.240265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.248115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.248904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.249396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.249731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.251605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.253742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.270949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1093/2183] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.270949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.281377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b199df-6de8-4434-b9dd-c00c1e333272 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.282422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1091/2184] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/projects => generated 399 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:17.283555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.366404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.367076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.367640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.368038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.368774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.371250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.373230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:17.379840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281e15-63fd-436a-8751-16be343de5bd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.388059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.389466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.390089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.390398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] PATH_INFO: `/v3/projects/0a8e115afb334317ba51e73582c1f2e3/users/ce36bd6c2eb546249ed1c89130e5a04e/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.401613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1094/2185] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:17.403023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.415682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8e115afb334317ba51e73582c1f2e3, user_id=ce36bd6c2eb546249ed1c89130e5a04e, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.416445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf043b8-79da-4a4b-93d4-a08bfaa1f6b6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.418621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.423449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.424456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.424914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.425599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.426522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.428730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd02547-49af-4a8b-9d86-c39239aa152f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.452115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1092/2186] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:17 2022] PUT /identity/v3/projects/0a8e115afb334317ba51e73582c1f2e3/users/ce36bd6c2eb546249ed1c89130e5a04e/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:17.454745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.461840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1095/2187] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/users => generated 300 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:17.463275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.463783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.464288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.502796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] MFA Rules not processed for user `ce36bd6c2eb546249ed1c89130e5a04e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:17.504307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:17.558614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:17.569852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1a2c-8471-4bf5-8ad6-f2f93fc61b22 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.580716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.581406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.581725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.582002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.582808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.584657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.596839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1096/2188] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:17.597882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.613550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1093/2189] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.613550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.613848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d103e-7a2e-453b-9c57-8252d567184a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.615451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:17.618127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.619023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.619295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.619632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.620396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.622018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.649759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1097/2190] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.650653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.670510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eb475-ad69-4bbd-a64f-f26b916345e5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.675737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.676403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.676693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.679050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.680142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.681912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.694933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1098/2191] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.696749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.708488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f4a9-bfc7-4558-9ca4-91447d8ba75f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.720276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6602c-12ee-4c11-8227-2ce8a21ed690 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.723863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.724303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.724941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.725059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.725230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.725514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.725850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.726282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.727485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.729801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.737322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.739067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.754616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1099/2192] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:17.757383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.759378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1094/2193] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:17.760999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:17.783542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11c80-958b-472a-8866-d79a03962870 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.784257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27b507-6fce-4cf6-af68-45f404df7ad2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:17.794017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.795064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:17.796459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.796748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.797032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] PATH_INFO: `/v3/projects/3f0a815bda3a4b55a130e065e54335dc/users/07bdb91d7b634451ae1d8c5f621f149a/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.798263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.798575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.798857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] PATH_INFO: `/v3/projects/b1b4f44a26374ae6881228530969c8cc/users/10801ebab1774c3482b0eb0a421a7451/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.829888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0a815bda3a4b55a130e065e54335dc, user_id=07bdb91d7b634451ae1d8c5f621f149a, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.832055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b4f44a26374ae6881228530969c8cc, user_id=10801ebab1774c3482b0eb0a421a7451, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:17.835549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6b4b2-ae7d-43ff-a336-fecc70254f02 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.839650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786c5dd-195b-42b3-88fb-5bf8e3019311 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:17.869912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1095/2194] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:17 2022] PUT /identity/v3/projects/b1b4f44a26374ae6881228530969c8cc/users/10801ebab1774c3482b0eb0a421a7451/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:17.870179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1100/2195] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:17 2022] PUT /identity/v3/projects/3f0a815bda3a4b55a130e065e54335dc/users/07bdb91d7b634451ae1d8c5f621f149a/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:17.881980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.882465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.882880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.933616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] MFA Rules not processed for user `07bdb91d7b634451ae1d8c5f621f149a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:17.935504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:17.950225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:17.950620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:17.950926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:17.990239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] MFA Rules not processed for user `10801ebab1774c3482b0eb0a421a7451`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:17.991410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:18.019173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:18.057916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1096/2196] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:18.059084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.129085 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1101/2197] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:16:18.142587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.145280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:18.206356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ce063-6e90-476c-a31c-972e80b40536 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.214389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.215903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.215903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.215903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.219062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.222528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.241496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f8de-2366-4c6e-a981-081ab67a5e5e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.246330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.247041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.247331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.247633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.261309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.263616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.269786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1097/2198] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:18 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:18.273930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.275332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1102/2199] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:18.368591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476371d-ea76-4bcf-8013-91753c6b9e32 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.373794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.374669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.375059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.375720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.376630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.383327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.414418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1098/2200] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:18 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:18.418061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.437703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2741-6f8b-427d-84ae-061103c96cd4 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.443642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.444363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.444716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.445070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.445881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.448099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.462415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1103/2201] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:18.463991 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.480487 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26269152-4057-4c34-81e5-664841b274cd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.486655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.487809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.488108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.488390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] PATH_INFO: `/v3/projects/cb3c3959d00d4378b06b7399de4d7db6/users/e112e637ed514f7394176795332debeb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.512617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3c3959d00d4378b06b7399de4d7db6, user_id=e112e637ed514f7394176795332debeb, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.516136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.538796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1099/2202] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:18 2022] PUT /identity/v3/projects/cb3c3959d00d4378b06b7399de4d7db6/users/e112e637ed514f7394176795332debeb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:18.541965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.553784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2064b-6817-4b63-b5fd-55125eb165bb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.564161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:18.663884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.664638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.664925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.665202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.665946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.667735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cca36-fc77-471e-a10f-abc2c31fdc11 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.679614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1104/2203] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:18.683449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.697691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac2010-14ac-4534-ad41-c9076949d581 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.703008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.704064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.704350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.704631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] PATH_INFO: `/v3/projects/cb3c3959d00d4378b06b7399de4d7db6/users/e112e637ed514f7394176795332debeb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.706830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3c3959d00d4378b06b7399de4d7db6, user_id=e112e637ed514f7394176795332debeb, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:18.710346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:18.728532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1100/2204] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:18 2022] PUT /identity/v3/projects/cb3c3959d00d4378b06b7399de4d7db6/users/e112e637ed514f7394176795332debeb/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:18.740094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae369e-e2b0-46c8-ad77-3d4d8ffb675c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.740461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae369e-e2b0-46c8-ad77-3d4d8ffb675c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.740752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae369e-e2b0-46c8-ad77-3d4d8ffb675c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:18.772240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ccae369e-e2b0-46c8-ad77-3d4d8ffb675c None None] MFA Rules not processed for user `e112e637ed514f7394176795332debeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:18.773561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ccae369e-e2b0-46c8-ad77-3d4d8ffb675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:18.864816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1105/2205] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:18.877246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:18.909629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:18.948405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41915b1-50c6-4400-b38e-cfca81de95b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:18.952994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:18.953863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:18.954841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:18.954997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:19.053761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:19.056462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363ac18-0a07-4a57-8f92-8e3e716e1b2d service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:19.067393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1101/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:21.407030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:21.407651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:21.407651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:21.428516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] MFA Rules not processed for user `ce36bd6c2eb546249ed1c89130e5a04e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:21.430116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:21.512758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1106/2207] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:23.415111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de69a-703c-45b5-9afb-f8cee0bc569d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:23.415694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de69a-703c-45b5-9afb-f8cee0bc569d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:23.415694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de69a-703c-45b5-9afb-f8cee0bc569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:23.438505 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0c2de69a-703c-45b5-9afb-f8cee0bc569d None None] MFA Rules not processed for user `031fb205335f407bb2d275476b98fe24`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:23.440060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0c2de69a-703c-45b5-9afb-f8cee0bc569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:23.523626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1102/2208] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:29.448288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:29.449843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:29.494400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:29.524409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70766605-c33b-4c04-b4ac-b192879b633e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:29.529670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:29.530645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:29.530915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:29.531194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] PATH_INFO: `/v3/users/b30d48d8025444c59df61ec85069b552` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:29.532471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d48d8025444c59df61ec85069b552)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:29.534507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:29.586833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d48d8025444c59df61ec85069b552'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:29.596011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d48d8025444c59df61ec85069b552'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:29.596698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d48d8025444c59df61ec85069b552'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:29.604808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d48d8025444c59df61ec85069b552'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:29.618561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d48d8025444c59df61ec85069b552'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:29.625452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1107/2209] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:29 2022] DELETE /identity/v3/users/b30d48d8025444c59df61ec85069b552 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:29.632124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:29.632647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:29.632734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:29.659592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:29.660834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:29.730183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1103/2210] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:29.743698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:29.745952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:29.828413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcc16e-f149-437f-b0c2-ac3e61b276a1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:29.833027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:29.834045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:29.834615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:29.835195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:29.849806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:29.851765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e17c-9e28-404a-acaa-e126f41ab5a2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:29.882459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1108/2211] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:30.227929 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:30.229595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:30.275352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:30.287921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:30.288222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:30.288555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:30.309724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:30.311083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:30.327799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7d869-4305-4789-90da-5f8bd533d5ae None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:30.332903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:30.334453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:30.334977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:30.335314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:30.337423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:30.403557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1109/2212] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:16:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:16:30.420960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:30.422776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:30.429585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:30.455193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:30.457544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:30.473358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1104/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:30.569616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7885-d47d-4426-b7b5-06c559c16906 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:30.600113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:30.600812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:30.601097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:30.601377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:30.617132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:30.618846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697a680-4e11-4a67-bea7-3fbfde327c20 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:30.642878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1110/2214] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:16:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 04:16:31.090106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:31.091781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:31.126137 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c76bbc-2384-489c-8389-73b5fc55e4b0 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:31.142410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:31.147065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:31.149382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:31.150214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] PATH_INFO: `/v3/users/10801ebab1774c3482b0eb0a421a7451` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:31.152544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] RBAC: Authorizing `identity:delete_user(user_id=10801ebab1774c3482b0eb0a421a7451)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:31.157974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:31.301919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10801ebab1774c3482b0eb0a421a7451'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.343705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10801ebab1774c3482b0eb0a421a7451'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.362742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10801ebab1774c3482b0eb0a421a7451'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.402022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10801ebab1774c3482b0eb0a421a7451'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.403344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10801ebab1774c3482b0eb0a421a7451'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.425248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1105/2215] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:31 2022] DELETE /identity/v3/users/10801ebab1774c3482b0eb0a421a7451 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:31.427114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:31.427522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:31.428444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:31.472277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:31.481039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:31.558589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:31.560107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:31.673875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1111/2216] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:31.694272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:31.696169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:31.728851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:31.776180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40823394-d231-41aa-b17a-913aa54e711a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:31.782491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:31.784106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:31.785036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:31.785364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] PATH_INFO: `/v3/projects/1288dfde91b340159ac74714c1ac820b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:31.786891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1288dfde91b340159ac74714c1ac820b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:31.788867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:31.792862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b49b0-ab62-4421-a317-9817d34fcca3 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:31.797754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:31.798616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:31.799007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:31.799471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:31.814608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:31.817959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:31.832000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1112/2217] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:31.939695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1288dfde91b340159ac74714c1ac820b', 'request_id': 'req-f63b11b8-7b67-4a28-8373-5b1819caa21d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.959610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1288dfde91b340159ac74714c1ac820b', 'request_id': 'req-f63b11b8-7b67-4a28-8373-5b1819caa21d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:31.986631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1288dfde91b340159ac74714c1ac820b', 'request_id': 'req-f63b11b8-7b67-4a28-8373-5b1819caa21d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.033276 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] The token cache is being invalidate because project 1288dfde91b340159ac74714c1ac820b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:32.034706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.035944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.043778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1106/2218] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:31 2022] DELETE /identity/v3/projects/1288dfde91b340159ac74714c1ac820b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:32.049026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:32.066018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff46a6-c907-4d7a-857e-30defc1040e2 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:32.156630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:32.211047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:32.212114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:32.212424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:32.212699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] PATH_INFO: `/v3/users/d02950ad806c41eeb67d849c787bdca7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:32.215597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02950ad806c41eeb67d849c787bdca7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:32.217717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:32.312539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02950ad806c41eeb67d849c787bdca7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.326961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02950ad806c41eeb67d849c787bdca7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.327671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02950ad806c41eeb67d849c787bdca7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.344646 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02950ad806c41eeb67d849c787bdca7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.379247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02950ad806c41eeb67d849c787bdca7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:32.392167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1113/2219] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:32 2022] DELETE /identity/v3/users/d02950ad806c41eeb67d849c787bdca7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:32.665766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:32.672851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:32.717380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b11b8-7b67-4a28-8373-5b1819caa21d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:32.843009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:32.844100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:32.844400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:32.844681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] PATH_INFO: `/v3/projects/b1b4f44a26374ae6881228530969c8cc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:32.847133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b4f44a26374ae6881228530969c8cc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:32.849786 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:33.018272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b4f44a26374ae6881228530969c8cc', 'request_id': 'req-14bac950-d79b-4c7e-a9ce-71612a3d8217'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.030086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b4f44a26374ae6881228530969c8cc', 'request_id': 'req-14bac950-d79b-4c7e-a9ce-71612a3d8217'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.050840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b4f44a26374ae6881228530969c8cc', 'request_id': 'req-14bac950-d79b-4c7e-a9ce-71612a3d8217'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.098559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] The token cache is being invalidate because project b1b4f44a26374ae6881228530969c8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:33.100584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.101522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14bac950-d79b-4c7e-a9ce-71612a3d8217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.104864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1107/2220] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:32 2022] DELETE /identity/v3/projects/b1b4f44a26374ae6881228530969c8cc => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:33.177582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:33.211357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:33.256319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e48ca9-33aa-4f06-9331-153a8993292b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:33.408863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:33.409913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:33.410294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:33.410691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] PATH_INFO: `/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:33.412437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff0d0400f884ba4aeff5cd3a617af43)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:33.414983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:33.536180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff0d0400f884ba4aeff5cd3a617af43', 'request_id': 'req-4d43aa1c-387b-4608-a4dd-0d24836cef0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.559512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff0d0400f884ba4aeff5cd3a617af43', 'request_id': 'req-4d43aa1c-387b-4608-a4dd-0d24836cef0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.563193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:33.582881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff0d0400f884ba4aeff5cd3a617af43', 'request_id': 'req-4d43aa1c-387b-4608-a4dd-0d24836cef0d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.628791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] The token cache is being invalidate because project 5ff0d0400f884ba4aeff5cd3a617af43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:33.629105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.629800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-4d43aa1c-387b-4608-a4dd-0d24836cef0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:33.632536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1114/2221] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:33 2022] DELETE /identity/v3/projects/5ff0d0400f884ba4aeff5cd3a617af43 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:33.639905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:33.640755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:33.640755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:33.671668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:33.708407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:33.709690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:33.903439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1108/2222] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:33.915960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:33.916509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:33.917127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:33.962594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:33.974598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:33.986659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1115/2223] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:33.991277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.006819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea336b-ebb9-4fa0-8170-a6db078fbacc None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.028575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.029303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.029589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.029870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.034949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:34.036689 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.041169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.078781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.082887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1109/2224] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:34.097831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cab2f9-e760-4cb1-9e1e-7cb9938ba3e2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.104654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.105439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.105807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.106046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.106887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.109929 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.155588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1116/2225] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:34 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:34.157450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.189486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc965e-8ebb-4162-a479-519fad5ddad2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.351016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.351858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.352191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.352626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.353379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.355411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.383703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1110/2226] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:34 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:34.387738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.404389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ac9d4-5be5-423d-9adb-196705107a3b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.409262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.410091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.410393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.410667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.411458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.413301 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.425134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1117/2227] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:34.429824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.440602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dc04d-15e8-435f-bdae-b2a08e1c08d7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.445401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.446091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.446401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.446736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.447484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.449238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101b529-bfe3-4c3f-b4c2-84c65fa6030f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.461869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1111/2228] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:34.466145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.478782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd068-8e1e-4fe2-bb2f-bbd061f1b141 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.483846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.484793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.485052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.485339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] PATH_INFO: `/v3/projects/9886a60f6a6945eaaf9ac17be8f8410e/users/f1f69b728f894f3b87eab33d9866b828/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.502242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9886a60f6a6945eaaf9ac17be8f8410e, user_id=f1f69b728f894f3b87eab33d9866b828, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.505068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.522789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1118/2229] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:34 2022] PUT /identity/v3/projects/9886a60f6a6945eaaf9ac17be8f8410e/users/f1f69b728f894f3b87eab33d9866b828/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:34.534815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.535338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.535771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.566357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] MFA Rules not processed for user `f1f69b728f894f3b87eab33d9866b828`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:34.568010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:34.644104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:34.648244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1112/2230] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:34.659229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:34.661026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:34.741434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9403e4-e90c-415c-bfcd-cb00947927e7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:34.745138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:34.745799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:34.746079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:34.746356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:34.756949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:34.758458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b223b0f-fa53-4b77-ae69-f883f32e6e42 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:34.769707 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1119/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:37.400015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:37.401796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:37.503237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e260b9-ab73-4c5d-ab5f-8c5a05232bbe None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:37.511970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:37.513018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:37.513450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:37.514168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] PATH_INFO: `/v3/users/07bdb91d7b634451ae1d8c5f621f149a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:37.515844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bdb91d7b634451ae1d8c5f621f149a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:37.517894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:37.604987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bdb91d7b634451ae1d8c5f621f149a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:37.618398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bdb91d7b634451ae1d8c5f621f149a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:37.633905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bdb91d7b634451ae1d8c5f621f149a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:37.653587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bdb91d7b634451ae1d8c5f621f149a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:37.654683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bdb91d7b634451ae1d8c5f621f149a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:37.662300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1113/2232] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:37 2022] DELETE /identity/v3/users/07bdb91d7b634451ae1d8c5f621f149a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:37.666885 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:37.667470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:37.667627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:37.689476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:37.690789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:37.793595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1120/2233] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:16:37.805032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:37.806520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:37.890699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa211458-c501-4d8f-9c11-4cc1364f24b4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:37.895614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:37.896798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:37.897176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:37.897589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:37.923584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:37.925093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:37.939749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1114/2234] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:38.261744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:38.263246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:38.275935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ba8b5-46a1-4b34-ad8a-3be912142f39 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:38.281217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:38.282110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:38.282417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:38.282695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] PATH_INFO: `/v3/projects/3f0a815bda3a4b55a130e065e54335dc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:38.284026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f0a815bda3a4b55a130e065e54335dc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:38.285967 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:38.310459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:38.339624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f0a815bda3a4b55a130e065e54335dc', 'request_id': 'req-6f40ef83-1b3a-4cf3-adae-afba1d95e491'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.345909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f0a815bda3a4b55a130e065e54335dc', 'request_id': 'req-6f40ef83-1b3a-4cf3-adae-afba1d95e491'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.354998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f0a815bda3a4b55a130e065e54335dc', 'request_id': 'req-6f40ef83-1b3a-4cf3-adae-afba1d95e491'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.366845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] The token cache is being invalidate because project 3f0a815bda3a4b55a130e065e54335dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:38.366955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.367610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.369181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1121/2235] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:38 2022] DELETE /identity/v3/projects/3f0a815bda3a4b55a130e065e54335dc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:38.373335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:38.383071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1c852-f4cc-43d5-87a0-e225ba18bad6 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:38.401034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:38.441344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:38.442267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:38.442679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:38.442954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] PATH_INFO: `/v3/users/e112e637ed514f7394176795332debeb` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:38.444176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e112e637ed514f7394176795332debeb)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:38.446101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:38.493868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e112e637ed514f7394176795332debeb'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.506514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e112e637ed514f7394176795332debeb'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.515828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e112e637ed514f7394176795332debeb'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.523809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e112e637ed514f7394176795332debeb'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.524471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-942cdf59-2c21-40ae-9af5-65a42f761eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e112e637ed514f7394176795332debeb'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:38.531253 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1115/2236] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:38 2022] DELETE /identity/v3/users/e112e637ed514f7394176795332debeb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:38.900290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:38.901829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:38.916147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40ef83-1b3a-4cf3-adae-afba1d95e491 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:38.976062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:38.977012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:38.977271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:38.977546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] PATH_INFO: `/v3/projects/cb3c3959d00d4378b06b7399de4d7db6` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:38.978835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3c3959d00d4378b06b7399de4d7db6)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:38.980735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.040789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3c3959d00d4378b06b7399de4d7db6', 'request_id': 'req-905aa10b-b405-490e-b3ba-b8f985e538ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:39.047810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3c3959d00d4378b06b7399de4d7db6', 'request_id': 'req-905aa10b-b405-490e-b3ba-b8f985e538ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:39.058261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3c3959d00d4378b06b7399de4d7db6', 'request_id': 'req-905aa10b-b405-490e-b3ba-b8f985e538ae'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:39.076091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] The token cache is being invalidate because project cb3c3959d00d4378b06b7399de4d7db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:39.076531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:39.077721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-905aa10b-b405-490e-b3ba-b8f985e538ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:39.081829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1122/2237] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:38 2022] DELETE /identity/v3/projects/cb3c3959d00d4378b06b7399de4d7db6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:39.087641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.087934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.088215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.110240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:39.111882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:39.180171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1116/2238] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:39.189138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.189504 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.189786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.209575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:39.210841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:39.219421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1123/2239] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:39.226338 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.242995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e26e-51cb-4cf7-884f-7cac9aaa4319 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.249835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.250633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.250960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.251313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.252240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:39.252594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.254964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.272531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1117/2240] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:39.278079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.288956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b24481-569b-4bab-b416-d3c7b9c2de6e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.295615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.296279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.296835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.297107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.297815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.301158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.347706 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1124/2241] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:39 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:39.353323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.368752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f672-1459-43f5-89e4-f24278830c65 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.406727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:39.447447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.448362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.448641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.448921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.449645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.451994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.480125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1118/2242] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:39 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:39.485277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.499817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990ea11-d559-45f7-bf08-1b6b4bc16971 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.504848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.505614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.505895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.506165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.506973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.508655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.511453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:39.526686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1125/2243] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:16:39.528890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.542079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e13a-3efe-4e33-9f3a-6db14c60d500 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.547393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.548179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.549316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.549686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.550547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.552906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eeb50-0da4-45eb-8883-c6987cd2eae6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.569516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1119/2244] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:39.573733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.601158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df9e4-4236-4942-9a71-c8408fc9ca98 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.613364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.615274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.615698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.616089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] PATH_INFO: `/v3/projects/0381c85f82cf4fdb81969ef3d4084e84/users/6b930b5b727f4fd78dae66841fbd4303/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.640006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0381c85f82cf4fdb81969ef3d4084e84, user_id=6b930b5b727f4fd78dae66841fbd4303, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.642972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:39.658909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1126/2245] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:39 2022] PUT /identity/v3/projects/0381c85f82cf4fdb81969ef3d4084e84/users/6b930b5b727f4fd78dae66841fbd4303/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:39.672893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.673200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.673512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.709106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] MFA Rules not processed for user `6b930b5b727f4fd78dae66841fbd4303`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:39.711164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:39.842272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1120/2246] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:39.850895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:39.853074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:39.974251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f495-a9ac-4c00-8607-ad964fa799a0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:39.974251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:39.974251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:39.974251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:39.975315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:39.984480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:39.985938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7777-9da6-4659-b072-05069ca1ec4c service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:40.002360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1127/2247] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:40.023407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:40.123968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0aaf6-16bd-4856-81d3-654b686d2e1b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:40.128917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:40.129845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:40.130161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:40.130478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] PATH_INFO: `/v3/users/6b930b5b727f4fd78dae66841fbd4303` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:40.131784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b930b5b727f4fd78dae66841fbd4303)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:40.133738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:40.206992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:40.221675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b930b5b727f4fd78dae66841fbd4303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.238394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b930b5b727f4fd78dae66841fbd4303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.250938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b930b5b727f4fd78dae66841fbd4303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.273184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b930b5b727f4fd78dae66841fbd4303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.284043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b930b5b727f4fd78dae66841fbd4303'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.302079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1121/2248] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:40 2022] DELETE /identity/v3/users/6b930b5b727f4fd78dae66841fbd4303 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:40.307423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:40.307902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:40.308373 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:40.336281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:40.337976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:40.441019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1128/2249] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:40.445472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:40.447122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:40.507719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c491-7ccb-41fb-bbcc-69db43aac6fd admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:40.511336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:40.512000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:40.512284 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:40.512558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:40.523400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:40.524904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caae9b-18e8-4e5d-82f1-94dc4e0b773e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:40.534888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1122/2250] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:40.866832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:40.868310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:40.890899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d2e37-3829-40d8-8f2a-268f1f757edc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:40.904217 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:40.905383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:40.905668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:40.905946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] PATH_INFO: `/v3/projects/0381c85f82cf4fdb81969ef3d4084e84` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:40.907432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0381c85f82cf4fdb81969ef3d4084e84)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:40.909913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:40.963718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0381c85f82cf4fdb81969ef3d4084e84', 'request_id': 'req-58a31b9b-d4b3-483a-99ae-a6033abce5f1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.969878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0381c85f82cf4fdb81969ef3d4084e84', 'request_id': 'req-58a31b9b-d4b3-483a-99ae-a6033abce5f1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.978199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0381c85f82cf4fdb81969ef3d4084e84', 'request_id': 'req-58a31b9b-d4b3-483a-99ae-a6033abce5f1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.993555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:40.998466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] The token cache is being invalidate because project 0381c85f82cf4fdb81969ef3d4084e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:40.998816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:40.999500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-58a31b9b-d4b3-483a-99ae-a6033abce5f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:41.006818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1129/2251] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:40 2022] DELETE /identity/v3/projects/0381c85f82cf4fdb81969ef3d4084e84 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:41.008799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.009094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.009377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.034280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:41.035429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:41.108195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1123/2252] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:41.117195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.117595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.117951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.139307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:41.151363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:41.170003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1130/2253] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:41.174426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.185228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c11f5f-3067-440b-865e-9de398bdf84e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.190101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.190830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.191121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.191403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.192155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:41.192853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.194554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.207712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1124/2254] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 04:16:41.212187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.221556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd7814-8c03-4a84-9bfc-5262eb2c2949 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.226982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.227657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.227940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.228220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.228927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.230875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.260136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1131/2255] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:41 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:41.264802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.275784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b57ca6-23e1-4369-9699-6e3dc0f7a8d6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.323244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:41.339348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.340175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.340534 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.340889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.341834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.343940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.371024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1125/2256] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:41 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:41.372656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.384097 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e729f-58a8-4ff1-bfd7-63d8e6d75bb0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.389156 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.389871 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.390160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.390505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.391374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.393396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.405981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1132/2257] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:41.410099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.421498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fda27-3553-4a56-aee8-f61796580d47 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.426591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.427618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.427878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.428162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.428911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.430569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35928c2-a1a4-4ca3-b39a-78dde314fc94 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.443827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1126/2258] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:16:41.447017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.458614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7ab2d-4880-48f3-9bda-d893c25ef893 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.463842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.465306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.465624 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.466061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] PATH_INFO: `/v3/projects/4c609e3388234c5f847b4537dbce2f28/users/25f4e322568b4fddb9acf5c0fe368908/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.486041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c609e3388234c5f847b4537dbce2f28, user_id=25f4e322568b4fddb9acf5c0fe368908, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.488971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.527900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1133/2259] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:41 2022] PUT /identity/v3/projects/4c609e3388234c5f847b4537dbce2f28/users/25f4e322568b4fddb9acf5c0fe368908/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:41.538942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925bfe-a89a-4494-9795-6191beeb1b13 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.539528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925bfe-a89a-4494-9795-6191beeb1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.539528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925bfe-a89a-4494-9795-6191beeb1b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.591643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-11925bfe-a89a-4494-9795-6191beeb1b13 None None] MFA Rules not processed for user `25f4e322568b4fddb9acf5c0fe368908`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:41.593040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-11925bfe-a89a-4494-9795-6191beeb1b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:41.738937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1127/2260] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:41.751341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:41.753182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:41.819902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b39be8-b86e-45a0-81c4-12ac78b6f872 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:41.823751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:41.824407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:41.824691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:41.824970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:41.833896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:41.835362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799acd2-41ac-4548-8bf8-f33013e7ba19 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:41.843959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1134/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:16:51.734306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:51.840099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:58.195623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.196115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.196115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.197611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1128/2262] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:16:58.208186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.208616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.208791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.235204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:58.236935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:58.287290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:58.292656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:58.332540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1135/2263] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:16:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:58.344927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:58.346689 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:58.360475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d23bc-2e7c-40d6-9dbb-b3065379704c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:58.365530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:58.366221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.366539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.366820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.381833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:58.383669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:58.384119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b738-4237-4086-b1bf-02eaccbdc4d3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:58.388665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:58.389867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.390149 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.390483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] PATH_INFO: `/v3/users/17986e47d6a34cd785ff91f0a000f2b5` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.391700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17986e47d6a34cd785ff91f0a000f2b5)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:58.393619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:58.395687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1136/2264] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:16:58.448175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17986e47d6a34cd785ff91f0a000f2b5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.461906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17986e47d6a34cd785ff91f0a000f2b5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.469992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17986e47d6a34cd785ff91f0a000f2b5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.479675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17986e47d6a34cd785ff91f0a000f2b5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.480467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f905eb46-da91-4fa1-98b7-553397bf2228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17986e47d6a34cd785ff91f0a000f2b5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.486998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1129/2265] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:16:58 2022] DELETE /identity/v3/users/17986e47d6a34cd785ff91f0a000f2b5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:58.769314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:58.780057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba32c24-0b20-41af-884f-c248aa6ba286 service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:58.847120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:58.848389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.848789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.849189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] PATH_INFO: `/v3/projects/f14e4e69a2db416caeef8036b55805b6` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.851285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14e4e69a2db416caeef8036b55805b6)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:58.854972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:58.914592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14e4e69a2db416caeef8036b55805b6', 'request_id': 'req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.921235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14e4e69a2db416caeef8036b55805b6', 'request_id': 'req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.930258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14e4e69a2db416caeef8036b55805b6', 'request_id': 'req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.941401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] The token cache is being invalidate because project f14e4e69a2db416caeef8036b55805b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:16:58.941695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.942327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-a4a0bea6-4a8e-4b31-ad05-c541dc064af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:16:58.943922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1137/2266] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:16:58 2022] DELETE /identity/v3/projects/f14e4e69a2db416caeef8036b55805b6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:16:58.950151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:58.950571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:58.950960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:58.973503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:58.974800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:59.045346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1130/2267] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:59.056854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.057142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.057424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.077218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:59.078470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:59.087920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1138/2268] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:16:59.092469 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.106224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40305d59-dc8c-44c4-8511-fa8d265d3644 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.113308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.114261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.114783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.115127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.116234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:16:59.116694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.119334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.135828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1131/2269] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:16:59.142178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.152175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5b8b-9892-40ac-9b86-286e3ece1b65 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.160685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.161378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.161658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.161924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.162681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.165600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.170195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:59.198401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1139/2270] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:16:59 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:59.202320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.212999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6942d-cf84-4ce6-9af7-9a2289b98658 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.217001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:16:59.275889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.276574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.276855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.277124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.277822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.279773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.302721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1132/2271] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:16:59 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:16:59.308221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.318788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284093-a0a0-43a0-8faa-0d7a63ffb2b1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.324430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.325084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.325364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.325636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.326341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.327990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.338731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1140/2272] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:59.343398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.357641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8081f-ebc8-46a3-abd5-b334e6aff068 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.363795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.364456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.364737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.365012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.365805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.367442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e67c2-ad12-41a3-bfd2-36bfddb63475 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.378740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1133/2273] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:16:59.382775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.393804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e67f3-4850-48fb-bcea-a317c8001e21 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.399057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.400126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.400496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.400827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] PATH_INFO: `/v3/projects/77e83a80c1b34e67958aca6d0a10ba8a/users/e43d573fcf384f65b7382f2d0a01ade4/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.418176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e83a80c1b34e67958aca6d0a10ba8a, user_id=e43d573fcf384f65b7382f2d0a01ade4, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.420834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.436167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1141/2274] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:16:59 2022] PUT /identity/v3/projects/77e83a80c1b34e67958aca6d0a10ba8a/users/e43d573fcf384f65b7382f2d0a01ade4/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:16:59.441933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2788-0900-4234-8c4b-24d7a61fbd97 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.442321 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2788-0900-4234-8c4b-24d7a61fbd97 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.442651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2788-0900-4234-8c4b-24d7a61fbd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.459739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-35ac2788-0900-4234-8c4b-24d7a61fbd97 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:16:59.460938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-35ac2788-0900-4234-8c4b-24d7a61fbd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:59.524256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1134/2275] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:16:59.534472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:16:59.536206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:16:59.591023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393c6c-474d-41b8-b232-c1a76783a5a3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:16:59.594875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:16:59.595506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:16:59.595787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:16:59.596099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:16:59.605444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:16:59.606909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:16:59.616402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1142/2276] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:04.286706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:04.287102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:04.294239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:04.365675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] MFA Rules not processed for user `e43d573fcf384f65b7382f2d0a01ade4`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:04.369435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:04.511939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1135/2277] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:04.527792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:04.529399 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:04.636983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4126e-ad3c-45e4-98f9-7dffb6161f72 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:04.651844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:04.653259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:04.653552 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:04.653821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:04.663411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:04.665437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f145f-6e39-41e3-ac89-bf37b22b8c5e service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:04.678246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1143/2278] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:06.446899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:06.448514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:06.463667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9548d-19f2-4330-8eac-cd41b1ff26f5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:06.468785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:06.469734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:06.470017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:06.470281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] PATH_INFO: `/v3/users/f1f69b728f894f3b87eab33d9866b828` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:06.472002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f69b728f894f3b87eab33d9866b828)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:06.473977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:06.486726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:06.533890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f69b728f894f3b87eab33d9866b828'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:06.549771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f69b728f894f3b87eab33d9866b828'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:06.558847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f69b728f894f3b87eab33d9866b828'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:06.568425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f69b728f894f3b87eab33d9866b828'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:06.570269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a1b4451b-3946-47c0-bc4a-553c6c75b80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f69b728f894f3b87eab33d9866b828'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:06.577135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1136/2279] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:06 2022] DELETE /identity/v3/users/f1f69b728f894f3b87eab33d9866b828 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:06.581545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:06.582068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:06.582160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:06.613372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:06.614431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:06.614732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:06.615029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:06.615166 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:06.639872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] MFA Rules not processed for user `e43d573fcf384f65b7382f2d0a01ade4`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:06.641116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:06.668474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:06.716091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1137/2280] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:17:06.742086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1144/2281] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:17:06.764707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:06.767269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:06.852196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fa64-e0ef-4aad-8a3f-bd166f6efe2e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:06.857317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:06.858127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:06.858535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:06.859167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:06.878944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:06.880910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2dd0a-4222-48ef-8d69-87c9e0b2e955 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:06.896112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1138/2282] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:07.862402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:07.868064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:07.913645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8480db-30d2-4fe1-bb5f-60e45777710a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:07.931411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:07.932232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:07.932518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:07.932797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] PATH_INFO: `/v3/projects/9886a60f6a6945eaaf9ac17be8f8410e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:07.934066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9886a60f6a6945eaaf9ac17be8f8410e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:07.936035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.035182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9886a60f6a6945eaaf9ac17be8f8410e', 'request_id': 'req-c080fc78-cd1e-4fae-95c2-b678afd5973e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:08.041983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9886a60f6a6945eaaf9ac17be8f8410e', 'request_id': 'req-c080fc78-cd1e-4fae-95c2-b678afd5973e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:08.057019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9886a60f6a6945eaaf9ac17be8f8410e', 'request_id': 'req-c080fc78-cd1e-4fae-95c2-b678afd5973e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:08.075634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] The token cache is being invalidate because project 9886a60f6a6945eaaf9ac17be8f8410e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:08.076005 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:08.076803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c080fc78-cd1e-4fae-95c2-b678afd5973e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:08.079106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1145/2283] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:07 2022] DELETE /identity/v3/projects/9886a60f6a6945eaaf9ac17be8f8410e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:08.084517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.085007 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.085108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.106874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:08.108123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:08.161832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:08.203391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1139/2284] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:08.213575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.213877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.214158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.244261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:08.246709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:08.260299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1146/2285] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:17:08.278509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.294591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692353a7-c826-4f67-9eb1-c3632088d68a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.300046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.300788 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.301079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.301363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.302761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:17:08.303081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.304733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.320411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1140/2286] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:17:08.325044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.335068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a027e4-0b73-48cd-9adb-43439ff425ff None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.343049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.343732 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.344022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.344447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.345152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.349114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.404703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1147/2287] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:08.409054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.427501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc93d2-ac76-401b-8516-0f0dfaf24912 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.502706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.503490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.503840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.504213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.504981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.507029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.529769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1141/2288] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:08.533781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.544954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e82f-1389-4797-bfd8-7417b78db08a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.550543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.551393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.551673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.551979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.552743 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.554336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.565771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1148/2289] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:08.569990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.581950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440bf33-8da0-4656-89b1-a959413ba44a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.589747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.590478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.590768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.591049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.591790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.593563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27fdd3-dde8-4159-843c-1d924838e407 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.610120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1142/2290] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:08.611595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.631946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90e34-d2ad-4242-925a-ce667a9f0973 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:08.646068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:08.646810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.646998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.647247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] PATH_INFO: `/v3/projects/c6fe8bc869224667b2ecbb4b5870efe1/users/840efee2ad004f768cfe65a44ffa7774/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.670328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fe8bc869224667b2ecbb4b5870efe1, user_id=840efee2ad004f768cfe65a44ffa7774, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:08.673453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:08.704744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1149/2291] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:08 2022] PUT /identity/v3/projects/c6fe8bc869224667b2ecbb4b5870efe1/users/840efee2ad004f768cfe65a44ffa7774/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:08.716238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:08.716529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:08.716808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:08.759671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] MFA Rules not processed for user `840efee2ad004f768cfe65a44ffa7774`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:08.760892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:08.889471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1143/2292] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:08.890734 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:08.976557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:09.070012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273bdd-f238-46a9-b43f-95688ea69481 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.075654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.078019 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.078353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.078680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.081653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.083907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.129559 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1150/2293] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:08 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:09.133397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.149920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:09.162694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b0301-874f-4fe6-90c1-fe6281e78119 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.223608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.224297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.224930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.224930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.225541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.227557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.248467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1144/2294] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:09 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:09.254725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.271428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2a26-17d2-4eff-9136-bdb316ce4c5e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.280130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.281142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.281508 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.281859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.283030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.285601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.301144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1151/2295] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:09.305334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.316154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd17fb-de6b-4fbe-8812-41944700dd2c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.322388 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.323068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.323361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.323648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.324373 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.325976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64844a51-ff4a-4b57-b6a1-4b039c79e079 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.336755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1145/2296] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:09.341037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.351820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504eab5-3d63-4012-9dfe-61f2b6245c16 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.357934 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.358872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.359160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.359426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] PATH_INFO: `/v3/projects/495a5552321d4b60ab833db9080d6897/users/26fe2a9677f24afa83ea7576f905a5bd/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.375248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495a5552321d4b60ab833db9080d6897, user_id=26fe2a9677f24afa83ea7576f905a5bd, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.377896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.393410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1152/2297] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:09 2022] PUT /identity/v3/projects/495a5552321d4b60ab833db9080d6897/users/26fe2a9677f24afa83ea7576f905a5bd/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:09.403811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.404097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.404368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.431862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] MFA Rules not processed for user `26fe2a9677f24afa83ea7576f905a5bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:09.433101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:09.505875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1146/2298] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:09.510770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.584563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f36cd7-24c7-436a-bce3-d61f94216dd6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.609669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.610378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.610699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.610947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.611661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.613590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.659376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1153/2299] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:09 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:09.663704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.685355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fc867-64e3-4d66-90fe-d07536605be1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.795559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.800228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.800743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.801098 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.801921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.804083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.826428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1147/2300] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:09 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:09.830875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.846672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b10f8-1679-4a1e-a490-402aee225b56 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.853642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.854423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.854765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.855032 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.855846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.857463 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.871519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1154/2301] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:09.875781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.882461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:09.889241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f17d5-d22e-4be5-a687-23270481b12c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:09.895518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:09.896919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:09.897387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:09.897895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] PATH_INFO: `/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152/users/378d402c401b43388766fd7dbccc0dd1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:09.929961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978d7ad9fb3d4ec3b0bd52777abe1152, user_id=378d402c401b43388766fd7dbccc0dd1, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:09.934445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:09.972546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1148/2302] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:09 2022] PUT /identity/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152/users/378d402c401b43388766fd7dbccc0dd1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:09.976451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:09.987250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f172669-96d8-4812-aa00-078f976fcd7c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:10.022473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:10.058133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:10.059854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:10.059854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:10.059854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:10.060162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:10.061788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3d155-9a4f-4077-a98a-535d9e612fb6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:10.084351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1155/2303] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:10.085960 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:10.096842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d14ff-01fe-48be-9237-bb646f421034 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:10.101787 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:10.102925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:10.103303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:10.103646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] PATH_INFO: `/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152/users/378d402c401b43388766fd7dbccc0dd1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:10.105679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] RBAC: Authorizing `identity:create_grant(project_id=978d7ad9fb3d4ec3b0bd52777abe1152, user_id=378d402c401b43388766fd7dbccc0dd1, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:10.108407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:10.119115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1149/2304] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:10 2022] PUT /identity/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152/users/378d402c401b43388766fd7dbccc0dd1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:10.130248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:10.130572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:10.130848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:10.158256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] MFA Rules not processed for user `378d402c401b43388766fd7dbccc0dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:10.159587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:10.235830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1156/2305] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:10.243057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:10.245113 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:10.326603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abfdd3-c1f2-4130-92f5-07c53026520f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:10.330823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:10.331478 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:10.331763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:10.332040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:10.395312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:10.396747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:10.406670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1150/2306] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:11.928976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:11.942517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0fd66-9ad6-4c87-8917-65e3d964c987 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:11.946475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:11.947132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:11.947417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:11.947690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:12.008812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:12.010269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7e80-2887-405a-83dc-78beb5e95be4 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:12.019737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1157/2307] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:12.662721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:12.671644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ef2a-2664-4ab0-b61a-97065c4a9738 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:12.675278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:12.675924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:12.676204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:12.676635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:12.686951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:12.689510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:12.700260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1151/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:20.501442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:20.501985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:20.501985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:20.519496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:20.520690 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:20.566757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:20.583701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1158/2309] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:17:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:20.607318 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:20.608820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:20.668260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:20.686476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63db6c-99db-4210-9562-3b193bef40b0 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:20.691682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:20.692334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:20.692625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:20.692900 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:20.701805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:20.703439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f14f0-055a-446f-96ac-11172e429c22 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:20.711994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1152/2310] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:17:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:17:20.993609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:21.074668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1051458-6c02-40cb-84b9-a3f1b0aa03a2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:21.080583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:21.081589 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.081937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.082278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] PATH_INFO: `/v3/users/840efee2ad004f768cfe65a44ffa7774` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.083586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=840efee2ad004f768cfe65a44ffa7774)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:21.085574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:21.152737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840efee2ad004f768cfe65a44ffa7774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:21.161357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840efee2ad004f768cfe65a44ffa7774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:21.162131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840efee2ad004f768cfe65a44ffa7774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:21.170203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840efee2ad004f768cfe65a44ffa7774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:21.187875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-110642d8-fdd7-4814-84b2-03fed5c8b348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840efee2ad004f768cfe65a44ffa7774'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:21.195375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1159/2311] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:20 2022] DELETE /identity/v3/users/840efee2ad004f768cfe65a44ffa7774 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:21.199906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.200240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.200584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.233758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:21.235237 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.302538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1153/2312] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:21.308104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.308400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.308683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.313153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:21.314706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.340771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:21.342649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.400750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1160/2313] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:17:21.401346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d29889-ea81-4b3c-89ac-4323b1df8a3b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:21.405945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:21.406744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.407106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.407453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.414325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:21.415794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.418777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:21.420372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:21.427694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:21.432949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1154/2314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:21.546503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fa599-31df-44ac-a393-d70e5ee30d6a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:21.565486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:21.566176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.566484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.576084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.586259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:21.588834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb819298-8659-48f8-87a8-b56fe873c68f service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:21.601282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1161/2315] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 04:17:21.879950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:21.882158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.923722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cae932-1f16-48d5-9f8d-232d431d6e71 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:21.930831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.931333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.931627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.937448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:21.938575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:21.939031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:21.939477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] PATH_INFO: `/v3/projects/c6fe8bc869224667b2ecbb4b5870efe1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:21.943461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fe8bc869224667b2ecbb4b5870efe1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:21.947656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:21.956614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:21.957984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:21.966077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1162/2316] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:21.979850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:21.995121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c391701-94c9-4ebe-99f1-a0c6273d187b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:22.001673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:22.002498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.002857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.003205 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.013076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:22.015767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:22.025770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1163/2317] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:17:22.032442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fe8bc869224667b2ecbb4b5870efe1', 'request_id': 'req-cab0738e-098a-409b-a354-9a537baaefe2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.034807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:22.042669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fe8bc869224667b2ecbb4b5870efe1', 'request_id': 'req-cab0738e-098a-409b-a354-9a537baaefe2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.051281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fe8bc869224667b2ecbb4b5870efe1', 'request_id': 'req-cab0738e-098a-409b-a354-9a537baaefe2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.072398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] The token cache is being invalidate because project c6fe8bc869224667b2ecbb4b5870efe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:22.072694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.073326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cab0738e-098a-409b-a354-9a537baaefe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.074980 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1155/2318] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:21 2022] DELETE /identity/v3/projects/c6fe8bc869224667b2ecbb4b5870efe1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:22.079173 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:22.090727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf48315-cd17-4504-855c-6a04ce338eda service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:22.154162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:22.155259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.155530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.155808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] PATH_INFO: `/v3/users/26fe2a9677f24afa83ea7576f905a5bd` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.157958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fe2a9677f24afa83ea7576f905a5bd)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:22.159127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:22.236381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fe2a9677f24afa83ea7576f905a5bd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.249409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fe2a9677f24afa83ea7576f905a5bd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.250152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fe2a9677f24afa83ea7576f905a5bd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.261671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fe2a9677f24afa83ea7576f905a5bd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.276818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fe2a9677f24afa83ea7576f905a5bd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.278539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:22.286576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1164/2319] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:22 2022] DELETE /identity/v3/users/26fe2a9677f24afa83ea7576f905a5bd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:22.385855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.386213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.386622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.406551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:22.407874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:22.467035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1156/2320] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:17:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:22.480685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:22.482455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:22.557134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6547667-5e1a-4c15-acd5-58336a474ea1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:22.563371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:22.564044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.564468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.564612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.573336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:22.574627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:22.576190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:22.577478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:22.587116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1165/2321] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:17:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:17:22.670285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd422-1d4f-4679-88c6-9b144adbe63d None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:22.674921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:22.675791 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.676074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.676351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] PATH_INFO: `/v3/projects/495a5552321d4b60ab833db9080d6897` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.677538 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] RBAC: Authorizing `identity:delete_project(project_id=495a5552321d4b60ab833db9080d6897)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:22.679521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:22.749539 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495a5552321d4b60ab833db9080d6897', 'request_id': 'req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.766871 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495a5552321d4b60ab833db9080d6897', 'request_id': 'req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.781963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495a5552321d4b60ab833db9080d6897', 'request_id': 'req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.816300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] The token cache is being invalidate because project 495a5552321d4b60ab833db9080d6897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:22.816999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.819120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:22.820847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1157/2322] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:22 2022] DELETE /identity/v3/projects/495a5552321d4b60ab833db9080d6897 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:22.824957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:22.844120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054866f-f8d0-4367-ab83-dffd5b4bd1dd service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:22.920944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:22.922056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:22.922392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:22.922658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] PATH_INFO: `/v3/users/378d402c401b43388766fd7dbccc0dd1` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:22.924242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=378d402c401b43388766fd7dbccc0dd1)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:22.926488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:23.019271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378d402c401b43388766fd7dbccc0dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.032071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378d402c401b43388766fd7dbccc0dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.034028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378d402c401b43388766fd7dbccc0dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.046143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378d402c401b43388766fd7dbccc0dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.083203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5c6a54b1-e9f4-4c63-9b78-0f60b6c3314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378d402c401b43388766fd7dbccc0dd1'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.090415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1166/2323] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:22 2022] DELETE /identity/v3/users/378d402c401b43388766fd7dbccc0dd1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:23.427556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:23.429203 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:23.452921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42f4a7-b83d-46c5-86c7-fd0fdefd713b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:23.472879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:23.531368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:23.532401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.532790 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.533162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] PATH_INFO: `/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.535231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] RBAC: Authorizing `identity:delete_project(project_id=978d7ad9fb3d4ec3b0bd52777abe1152)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:23.545122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:23.607654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978d7ad9fb3d4ec3b0bd52777abe1152', 'request_id': 'req-14f03651-6b70-4170-ad0f-6303aad5188d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.616837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978d7ad9fb3d4ec3b0bd52777abe1152', 'request_id': 'req-14f03651-6b70-4170-ad0f-6303aad5188d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.629156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978d7ad9fb3d4ec3b0bd52777abe1152', 'request_id': 'req-14f03651-6b70-4170-ad0f-6303aad5188d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.642511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] The token cache is being invalidate because project 978d7ad9fb3d4ec3b0bd52777abe1152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:23.642937 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.643677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-14f03651-6b70-4170-ad0f-6303aad5188d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:23.645542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1158/2324] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:23 2022] DELETE /identity/v3/projects/978d7ad9fb3d4ec3b0bd52777abe1152 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:23.654739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.655269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.655387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.669276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:23.716647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:23.718397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:23.786894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1167/2325] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:23.794783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.795075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.795327 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.815611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:23.817270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:23.825515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1159/2326] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:23.829881 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:23.840647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615835f0-f0ba-4054-abd8-5d6b717dd4c1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:23.845983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:23.846731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.847013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.847287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.848023 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:17:23.848330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:23.850308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:23.863840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1168/2327] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:17:23.867892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:23.878257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e297-a37c-4f59-b68f-1ca0aad5d665 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:23.883163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:23.883833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.884119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.884394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.885109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:23.887084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:23.917968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1160/2328] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:23 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:23.922312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:23.932817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23bd8b-96e2-4fe7-98f1-7b5b0afbcc04 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:23.992201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:23.992888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:23.993174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:23.993486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:23.994250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:23.996313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:24.020297 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1169/2329] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:23 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:24.024514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:24.035644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ff51-f39c-4b7a-bc0e-1394ce3be2e6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:24.040516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:24.042423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:24.042920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:24.043099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:24.043859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:24.046202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:24.057567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1161/2330] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:24.061876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:24.073377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f931c-4bbb-401a-9a0e-f43e27d5ea66 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:24.079420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:24.080303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:24.080719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:24.081113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:24.082096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:24.084045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14927ab9-01dd-4442-a4a8-3a2e16134a1f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:24.097141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1170/2331] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:24.101734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:24.113115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a626a-6559-489f-bb34-9721357550e2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:24.118847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:24.119926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:24.120473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:24.120855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] PATH_INFO: `/v3/projects/cbaaacda7cec4e87bcfb0c74f16bb5ec/users/8cd49062c283459189b275fd1b606d5e/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:24.146525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaaacda7cec4e87bcfb0c74f16bb5ec, user_id=8cd49062c283459189b275fd1b606d5e, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:24.149497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:24.174931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1162/2332] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:24 2022] PUT /identity/v3/projects/cbaaacda7cec4e87bcfb0c74f16bb5ec/users/8cd49062c283459189b275fd1b606d5e/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:24.194249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:24.194249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:24.194249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:24.225201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] MFA Rules not processed for user `8cd49062c283459189b275fd1b606d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:24.226561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:24.271238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:24.300239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1171/2333] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:24.312924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:24.314676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:24.363542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:24.376853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1805-ca94-4617-aace-763699714c18 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:24.381548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:24.382267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:24.382585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:24.382869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:24.392609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:24.394148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f62d6-8648-4e43-8685-48e83ebce7e7 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:24.403833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1163/2334] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:25.030640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:25.032133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.108405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556530-5829-4be2-b426-a5f94fcb6a41 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:25.112904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:25.113821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.114102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.114394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] PATH_INFO: `/v3/users/ce36bd6c2eb546249ed1c89130e5a04e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.115623 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce36bd6c2eb546249ed1c89130e5a04e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:25.117566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:25.165391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce36bd6c2eb546249ed1c89130e5a04e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.174266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce36bd6c2eb546249ed1c89130e5a04e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.175116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce36bd6c2eb546249ed1c89130e5a04e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.182809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce36bd6c2eb546249ed1c89130e5a04e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.196474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce36bd6c2eb546249ed1c89130e5a04e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.203499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1172/2335] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:25 2022] DELETE /identity/v3/users/ce36bd6c2eb546249ed1c89130e5a04e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:25.208395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.210835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.211277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.234291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:25.236008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.238180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:25.239383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.312436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:25.315912 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1164/2336] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:25.316514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc61d-cca5-41cc-b9a1-2ee133645c54 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:25.321639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:25.322567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.323674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.324230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] PATH_INFO: `/v3/users/25f4e322568b4fddb9acf5c0fe368908` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.326291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f4e322568b4fddb9acf5c0fe368908)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:25.329069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:25.329430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:25.331927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.409062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f4e322568b4fddb9acf5c0fe368908'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.413084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8f64c-4632-43d8-b940-12b093e06e97 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:25.418456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:25.420488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.420795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.421136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.423104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f4e322568b4fddb9acf5c0fe368908'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.424342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f4e322568b4fddb9acf5c0fe368908'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.434027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:25.436420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909c883-7424-48f1-a7ec-2f23ef8d6c5f service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:25.436547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f4e322568b4fddb9acf5c0fe368908'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.447992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1165/2337] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:25.456890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f4e322568b4fddb9acf5c0fe368908'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.464344 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1173/2338] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:25 2022] DELETE /identity/v3/users/25f4e322568b4fddb9acf5c0fe368908 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:25.469254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.469618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.469963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.491025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:25.492324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.567904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1166/2339] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:25.577607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:25.579462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.653833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e02dce-6e91-4eab-8b76-42f2187343ba admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:25.657652 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:25.658423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.658720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.658991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.668439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:25.669971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:25.691406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:25.692299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1174/2340] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:25.693095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:25.707286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebd96a-2bd2-40eb-a444-95f8d48ded98 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:25.714031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:25.715202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:25.716171 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:25.716695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] PATH_INFO: `/v3/projects/0a8e115afb334317ba51e73582c1f2e3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:25.718422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8e115afb334317ba51e73582c1f2e3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:25.720502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:25.750589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:25.816759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8e115afb334317ba51e73582c1f2e3', 'request_id': 'req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.833544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8e115afb334317ba51e73582c1f2e3', 'request_id': 'req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.842642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8e115afb334317ba51e73582c1f2e3', 'request_id': 'req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.878659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] The token cache is being invalidate because project 0a8e115afb334317ba51e73582c1f2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:25.879928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.882979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4b8be6b1-59d0-475d-a4f4-f3ee8af643b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:25.890414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1167/2341] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:25 2022] DELETE /identity/v3/projects/0a8e115afb334317ba51e73582c1f2e3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:26.282808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:26.284338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:26.299221 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569000ed-94b9-42ce-98d7-124539ae648a service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:26.360515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:26.361396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:26.361680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:26.361960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] PATH_INFO: `/v3/projects/4c609e3388234c5f847b4537dbce2f28` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:26.363201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c609e3388234c5f847b4537dbce2f28)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:26.365099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:26.388000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:26.413864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c609e3388234c5f847b4537dbce2f28', 'request_id': 'req-5496abe9-348c-4011-9910-2fed909e78a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:26.420452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c609e3388234c5f847b4537dbce2f28', 'request_id': 'req-5496abe9-348c-4011-9910-2fed909e78a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:26.429162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c609e3388234c5f847b4537dbce2f28', 'request_id': 'req-5496abe9-348c-4011-9910-2fed909e78a6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:26.440320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] The token cache is being invalidate because project 4c609e3388234c5f847b4537dbce2f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:26.440631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:26.441311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:26.442958 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1175/2342] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:26 2022] DELETE /identity/v3/projects/4c609e3388234c5f847b4537dbce2f28 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:33.597160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:33.597742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:33.597742 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:33.616890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] MFA Rules not processed for user `dca54bc220104e79aa727f30399f3411`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:33.618109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:33.681086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1168/2343] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:36.437875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:38.602537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:38.604074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:38.685263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496abe9-348c-4011-9910-2fed909e78a6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:38.690021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:38.690728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:38.691006 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:38.691286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:38.693154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:38.777737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:38.779503 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:38.794819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1176/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:43.674174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=40,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:44.943810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:44.945486 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:45.049930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1744-b5bc-43d2-b424-71fed228ba67 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:45.056932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:45.057605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:45.057886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:45.058163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:45.060019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:45.072985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:45.074952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f6615-2e26-45fa-bfa5-94cc4493dadc service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:45.086114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1169/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:48.219360 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:48.221084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:48.292853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be103-d224-4a6f-b3fa-21904e29ef53 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:48.308171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:48.309234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:48.309583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:48.309957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] PATH_INFO: `/v3/users/8cd49062c283459189b275fd1b606d5e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:48.311454 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd49062c283459189b275fd1b606d5e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:48.313608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:48.364414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd49062c283459189b275fd1b606d5e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:48.368180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:48.373012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd49062c283459189b275fd1b606d5e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:48.373770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd49062c283459189b275fd1b606d5e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:48.383099 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd49062c283459189b275fd1b606d5e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:48.395746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd49062c283459189b275fd1b606d5e'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:48.401659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1177/2346] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:17:48 2022] DELETE /identity/v3/users/8cd49062c283459189b275fd1b606d5e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:48.406298 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:48.406801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:48.406915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:48.425185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:48.426423 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:48.489924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1170/2347] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:48.500466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:48.501984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:48.565031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330bda6-a68a-4265-8321-827f74cddc9e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:48.568643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:48.569284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:48.569566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:48.569838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:48.578980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:48.580432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4818517-f0a8-407a-a196-00d829417d0a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:48.589986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1178/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:49.020004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.021628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:49.036746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38650100-76bb-47a0-93fd-52d22b536502 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.046258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.046603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.046888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.047179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] PATH_INFO: `/v3/projects/cbaaacda7cec4e87bcfb0c74f16bb5ec` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.048795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaaacda7cec4e87bcfb0c74f16bb5ec)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.050932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.103335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaaacda7cec4e87bcfb0c74f16bb5ec', 'request_id': 'req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:49.109867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaaacda7cec4e87bcfb0c74f16bb5ec', 'request_id': 'req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:49.116033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaaacda7cec4e87bcfb0c74f16bb5ec', 'request_id': 'req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:49.118792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:49.132391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] The token cache is being invalidate because project cbaaacda7cec4e87bcfb0c74f16bb5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:17:49.132715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:49.133356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-678d4e50-84b5-447c-9a9a-0a3e5263d5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:17:49.134977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1171/2349] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:17:49 2022] DELETE /identity/v3/projects/cbaaacda7cec4e87bcfb0c74f16bb5ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:49.140381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc8d05-c511-4207-a396-0122462f691a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.140728 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc8d05-c511-4207-a396-0122462f691a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.141017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc8d05-c511-4207-a396-0122462f691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.159677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-25fc8d05-c511-4207-a396-0122462f691a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:49.160894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-25fc8d05-c511-4207-a396-0122462f691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:49.223278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1179/2350] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:17:49.231337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.231654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.231927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.250176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:49.251472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:49.259550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1172/2351] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:49.264376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc8d05-c511-4207-a396-0122462f691a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.274447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc8d05-c511-4207-a396-0122462f691a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.279029 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.279771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.280055 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.280331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.281056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:17:49.281361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.283084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.295357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1180/2352] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:17:49.299515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.309653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4648625-3f8f-4a81-9869-208ac70a5e8a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.315003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.315746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.316672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.317051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.317820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.319757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.352128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1173/2353] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:49 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:49.353581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.363863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01cfd9-8745-4b30-89d0-25ff49e2ad32 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.423742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.424424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.424708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.424987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.426194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.427830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.434669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:49.453067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1181/2354] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:49 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:49.454489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.465072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a97e8-d451-4c76-864a-713c6d2b5c4d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.470561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.471221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.471499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.471761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.472490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.474087 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.487763 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1174/2355] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:49.491833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.504425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19e3-0cd7-45a3-a322-169d3914b69d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.509814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.510488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.510774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.511049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.511781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.513388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a25b-d707-4939-b9ff-2b4c4249b233 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.526654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1182/2356] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:49.531156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.543150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312127e-b599-434a-ab2d-c3377158d78f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.550992 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.551885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.552167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.552407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] PATH_INFO: `/v3/projects/b622ab692a73432784c63b9ad3290e72/users/34ec54486c9d404896f832b87f99bc11/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.568659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b622ab692a73432784c63b9ad3290e72, user_id=34ec54486c9d404896f832b87f99bc11, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.571359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.586760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1175/2357] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:49 2022] PUT /identity/v3/projects/b622ab692a73432784c63b9ad3290e72/users/34ec54486c9d404896f832b87f99bc11/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:49.592033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f5568-767a-4f50-a83c-043b9b289a28 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.592352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f5568-767a-4f50-a83c-043b9b289a28 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.592633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f5568-767a-4f50-a83c-043b9b289a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.611943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-421f5568-767a-4f50-a83c-043b9b289a28 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:49.613056 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-421f5568-767a-4f50-a83c-043b9b289a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:49.673855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1183/2358] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:49.683696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:49.685560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:49.745209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63509090-25f3-4abd-bc65-45e0e305cfd7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:49.749772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:49.750421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:49.750760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:49.751032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:49.761335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:49.762802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:49.773547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1176/2359] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:17:53.896348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:53.896810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:53.896891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:53.927821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] MFA Rules not processed for user `34ec54486c9d404896f832b87f99bc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:53.929180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:54.024785 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1184/2360] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:54.029336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.039592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a33c8-ce6f-45cd-ac48-bb8e016dd8d3 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.044458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.045326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.045934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.046443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.047330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.049499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.081736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1177/2361] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:17:54 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:17:54.083540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.095998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408436c4-86ae-41a9-b6cf-f10c07cde651 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.173252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.173955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.174237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.174550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.175326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.177983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.197651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:54.207131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1185/2362] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:17:54 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:17:54.211026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.227556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a21-5fa6-467f-af0f-f1ee0266e6ad admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.232692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.233444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.233731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.234005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.234831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.236797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.248941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1178/2363] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:17:54.254282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.267494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621619b6-3d2e-4a34-86b9-5d42a94c21bb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.272647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.275537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.275947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.276588 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] PATH_INFO: `/v3/projects/2585949985ce4eb199960e63235257cf/users/e6357d98d18f435c950fdc3fe2982c5d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.294094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2585949985ce4eb199960e63235257cf, user_id=e6357d98d18f435c950fdc3fe2982c5d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.297119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.316330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1186/2364] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:54 2022] PUT /identity/v3/projects/2585949985ce4eb199960e63235257cf/users/e6357d98d18f435c950fdc3fe2982c5d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:54.321242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.338567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d7481-233a-4712-80d2-c12cdc663134 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.401206 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.401881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.402167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.402521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.403261 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.404911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4c90-70b1-40dd-bde4-fe33a2802e9d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.406831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:17:54.422064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1179/2365] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:17:54.423647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:54.436637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444ae28-0377-4e11-b542-d2a84efe49c6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:54.443952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:54.444924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:54.445164 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:54.445447 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] PATH_INFO: `/v3/projects/2585949985ce4eb199960e63235257cf/users/e6357d98d18f435c950fdc3fe2982c5d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:54.447539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2585949985ce4eb199960e63235257cf, user_id=e6357d98d18f435c950fdc3fe2982c5d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:54.450199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:54.463489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1187/2366] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:17:54 2022] PUT /identity/v3/projects/2585949985ce4eb199960e63235257cf/users/e6357d98d18f435c950fdc3fe2982c5d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:17:59.468632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb55d-fcf5-4114-926a-25ac0bccec91 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:59.469170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb55d-fcf5-4114-926a-25ac0bccec91 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:59.469623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb55d-fcf5-4114-926a-25ac0bccec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:59.498633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-511cb55d-fcf5-4114-926a-25ac0bccec91 None None] MFA Rules not processed for user `e6357d98d18f435c950fdc3fe2982c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:17:59.499899 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-511cb55d-fcf5-4114-926a-25ac0bccec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:59.569598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1180/2367] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:17:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:17:59.587218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:17:59.589154 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:17:59.699990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc418f-597f-40ec-badd-07a945f1bf9c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:17:59.706751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:17:59.707483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:17:59.707784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:17:59.708074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:17:59.772167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:17:59.773883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b675e-1144-4006-be81-54dd74b18983 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:17:59.784494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1188/2368] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:17:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:18:09.566452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=36 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:18:09.779701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:18:25.651190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:18:25.651733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:18:25.651733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:18:25.671393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] MFA Rules not processed for user `34ec54486c9d404896f832b87f99bc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:18:25.672584 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:18:25.680537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1181/2369] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:18:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:18:35.665535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:18:37.711622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:18:37.712112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:18:37.712206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:18:37.729532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] MFA Rules not processed for user `4041faa2919d4760ae10907070a10dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:18:37.730774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:18:37.787206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1189/2370] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:18:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:18:47.373775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:18:47.375593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:18:47.440386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd7202-2a0e-4212-a61d-e6909113999e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:18:47.444410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:18:47.445074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:18:47.445347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:18:47.445613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:18:47.456162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:18:47.457613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:18:47.468382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1182/2371] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:18:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:18:47.779749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=25 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:18:57.463820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=26 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:19:31.938937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:31.950146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1f460-8fa6-4d9c-88cd-c306e9953c5b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:31.954733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:31.955390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:31.955673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:31.955933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:31.957823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:31.972267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:31.973835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccec43-8eea-42bb-bccb-5f6ba2d36b3b service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:31.985410 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1190/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:33.179696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:33.189193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e6553-d895-4ffd-bf65-f930c0f364d6 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:33.193741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:33.194397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:33.194701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:33.194970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:33.197021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:33.210700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:33.212925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:33.224351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1183/2373] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:33.835136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:33.835629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:33.835725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:33.885371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:19:33.886612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:33.954739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1191/2374] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:19:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:19:34.005454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:34.007259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:34.103332 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bf19a-a069-4aa5-b3b4-b88c0e2fe9cf service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:34.108808 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:34.109554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:34.109836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:34.110112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:34.119033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:34.120625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:34.129267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1184/2375] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:19:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:19:39.124339 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:39.126285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:39.244968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92b30d-3627-4636-b03a-ca7bf7311535 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:39.248844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:39.249525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:39.249837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:39.250112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:39.252582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:39.325236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:39.327110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4231b5e-fc88-4210-a2ce-f1bc3c880640 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:39.334794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:19:39.367731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1192/2376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:44.125700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=47 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:19:49.340353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:19:49.500905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:49.511129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7353a9-839d-4a68-9978-ec25a11c02ea service swift] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:49.515668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:49.516314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.516605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.516884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.519085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:49.528964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:49.530444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ece691-3e73-4077-acf4-1d5db863721e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:49.538722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1185/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:49.647027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.647316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.647596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.648988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1193/2378] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:19:49.655072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67698c1-96de-4866-bf1b-b56c26da2de0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.655361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67698c1-96de-4866-bf1b-b56c26da2de0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.655638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67698c1-96de-4866-bf1b-b56c26da2de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.695836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f67698c1-96de-4866-bf1b-b56c26da2de0 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:19:49.697057 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f67698c1-96de-4866-bf1b-b56c26da2de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:49.707661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1186/2379] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:19:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:19:49.719328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:49.721500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:49.810185 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d8368-0d32-4dc7-941b-9f2bd7362123 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:49.814412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:49.815086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.815770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.815770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.827961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:49.829475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e97e7-dc30-433b-acec-582fb9b20a65 service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:49.841874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1194/2380] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:49.926956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0936-6be6-4ed2-b556-cacf50c524d3 None None] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.927242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0936-6be6-4ed2-b556-cacf50c524d3 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.927522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0936-6be6-4ed2-b556-cacf50c524d3 None None] PATH_INFO: `/` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.928909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1187/2381] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:19:49.934325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c037a6-2a2b-4d27-a675-ab9e455ac866 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.934704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c037a6-2a2b-4d27-a675-ab9e455ac866 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.934948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c037a6-2a2b-4d27-a675-ab9e455ac866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:49.953543 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-36c037a6-2a2b-4d27-a675-ab9e455ac866 None None] MFA Rules not processed for user `7e468089828e4180b8636e167d16cf8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:19:49.954866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-36c037a6-2a2b-4d27-a675-ab9e455ac866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:49.961381 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1195/2382] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:19:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:19:49.973796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d0936-6be6-4ed2-b556-cacf50c524d3 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:49.989872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d0936-6be6-4ed2-b556-cacf50c524d3 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:49.993989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:49.994674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:49.994961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:49.995238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:50.003831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:50.005300 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370afff-45ec-4e36-8b87-b5b75a29205f service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:50.014080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1188/2383] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:50.113151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded07c-ce50-47e0-9393-41e75fe4767e None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:50.113468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded07c-ce50-47e0-9393-41e75fe4767e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:50.113796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded07c-ce50-47e0-9393-41e75fe4767e None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:50.115231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1196/2384] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:19:50.120815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:50.121106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:50.121383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:50.145126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] MFA Rules not processed for user `789dd1683eac49d9b26609eb2903475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:19:50.146858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:50.161821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1189/2385] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:19:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:19:50.172841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded07c-ce50-47e0-9393-41e75fe4767e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:50.185773 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded07c-ce50-47e0-9393-41e75fe4767e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:50.190260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:50.190964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:50.191253 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:50.191535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:50.202039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:50.203515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681da53-453e-484d-a0b9-b018e17e585c service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:50.215028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1197/2386] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:50.281102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:50.283130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:50.360238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a4d44-b8b3-4ce2-a07b-7c7a60306e44 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:50.366410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:50.367023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:50.367307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:50.367590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:50.369713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:50.382047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:50.383699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:50.394352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1190/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:19:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:19:53.594644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:53.595187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:53.595187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:53.613350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:19:53.614654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:19:53.622505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1198/2388] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:19:53.635317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:19:53.647366 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31badbb-61c6-4980-b6ad-daccd6d8566b service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:19:53.652654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:19:53.653285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:19:53.653568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:19:53.653846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:19:53.662207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:19:53.663851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308438dc-0b05-45b2-9607-10356b41b3c7 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:19:53.672121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1191/2389] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:20:03.608184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=47 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:03.668777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=62 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:13.768841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:13.779349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2785191-f3c6-4672-91a5-529c99d8a35b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:13.783769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:13.784439 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:13.784720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:13.784995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:13.787182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:13.797660 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:13.799125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:13.807356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1199/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:15.894815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e591-87d2-4801-a929-b21604237357 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:15.895410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e591-87d2-4801-a929-b21604237357 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:15.895410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e591-87d2-4801-a929-b21604237357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:15.913856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9443e591-87d2-4801-a929-b21604237357 None None] MFA Rules not processed for user `dca54bc220104e79aa727f30399f3411`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:20:15.915299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9443e591-87d2-4801-a929-b21604237357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:15.925062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1192/2391] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:20:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:20:23.798805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:23.804493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:23.815997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ea56-c64b-49df-a755-675a5530658b service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:23.823333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:23.824101 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:23.824525 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:23.824870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:23.826876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:23.840847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:23.842484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:23.853895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1200/2392] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:25.009974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:25.010548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:25.010548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:25.030279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:20:25.031527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:25.039035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1193/2393] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:20:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:20:25.052026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:25.064024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eeb9f-a8e8-4bf2-b73f-e675f474573d service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:25.072260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:25.073084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:25.073569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:25.073858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:25.083486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:25.085061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:25.093304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1201/2394] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:20:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:20:35.024116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:35.090065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=12 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:37.202319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:37.204281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:37.221642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdb4ba-6f19-4966-8481-ffc1cb0f1992 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:37.227771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:37.228419 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:37.228698 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:37.228976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:37.230921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:37.241796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:37.243425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0a64d-c70f-4551-b629-12e64c3e284f service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:37.252453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1194/2395] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:41.585394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:41.599770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808060bb-2f9c-49b8-8eb9-63b74bd1a391 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:41.605804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:41.606502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:41.606788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:41.607210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:41.609340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:41.626667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:41.632791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:41.664756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1202/2396] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:47.248312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:50.416908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff4502-5152-4d95-a17f-21e97ec4ba7e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:50.417459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff4502-5152-4d95-a17f-21e97ec4ba7e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:50.417459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff4502-5152-4d95-a17f-21e97ec4ba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:50.435661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bdff4502-5152-4d95-a17f-21e97ec4ba7e None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:20:50.437019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bdff4502-5152-4d95-a17f-21e97ec4ba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:50.444797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1195/2397] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:20:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:20:50.468204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:50.480944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19090a2d-010a-4a65-9446-ad66d68e4532 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:50.487350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:50.488258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:50.488546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:50.488806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:50.498138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:50.500183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:50.511174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1203/2398] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:20:54.383093 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:54.383596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:54.383596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:54.402563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] MFA Rules not processed for user `4041faa2919d4760ae10907070a10dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:20:54.403903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:54.411403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1196/2399] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:20:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:20:55.001319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:55.011411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56ed96-b29c-404b-9dab-52f854d31a1f service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:55.016216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:55.016887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:55.017260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:55.017529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:55.019680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:55.030586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:55.032021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8d997-cf28-4e53-8341-ee14eab484ba service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:55.040430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1204/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:56.819215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:56.829842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ffbd7-1c42-4893-bf1e-2a1a2ab36557 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:56.833641 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:56.834324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:56.834639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:56.834919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:56.836839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:56.902319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:56.903777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b640d3-ab12-446b-ae70-8d1b5bb19e85 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:56.913572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1197/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:57.975254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:57.975833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:57.975945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] PATH_INFO: `/` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:57.977486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1205/2402] 10.222.0.16 () {54 vars in 921 bytes} [Mon Jul 4 04:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 04:20:57.987506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:57.988016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:57.988016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:58.026138 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] MFA Rules not processed for user `01abfdc5d30744c89a476de93f537305`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:20:58.027380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:58.037507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1198/2403] 10.222.0.16 () {58 vars in 1018 bytes} [Mon Jul 4 04:20:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:20:58.052293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:58.053293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:58.053498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:58.054973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:58.069421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36ab47-7678-49e1-afd1-17630c5d3cc6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:58.070451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c98362-9f76-434d-9e06-a57bf6071e2f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:58.074257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:58.075231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:58.075379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:58.075518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:58.075827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] PATH_INFO: `/v3/users/031fb205335f407bb2d275476b98fe24` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:58.076098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:58.076379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:58.076826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:58.077049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=031fb205335f407bb2d275476b98fe24)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:58.079017 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:58.085453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:58.087029 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:58.096739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1206/2404] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:20:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:20:58.130490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031fb205335f407bb2d275476b98fe24'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.143530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031fb205335f407bb2d275476b98fe24'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.152056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031fb205335f407bb2d275476b98fe24'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.158909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031fb205335f407bb2d275476b98fe24'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.159847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031fb205335f407bb2d275476b98fe24'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.165806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1199/2405] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:20:58 2022] DELETE /identity/v3/users/031fb205335f407bb2d275476b98fe24 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:20:58.349933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:58.361942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead56e6-47a7-4cb9-87c5-8b8927052b0f service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:58.421377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:58.422211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:58.422530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:58.422810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] PATH_INFO: `/v3/projects/c487ee37e54042d680eccb72b046bb55` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:58.424062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c487ee37e54042d680eccb72b046bb55)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:58.425980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:58.474727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c487ee37e54042d680eccb72b046bb55', 'request_id': 'req-2a0e5466-551d-4a84-a1f3-d17bcc59a709'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.480792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c487ee37e54042d680eccb72b046bb55', 'request_id': 'req-2a0e5466-551d-4a84-a1f3-d17bcc59a709'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.490401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c487ee37e54042d680eccb72b046bb55', 'request_id': 'req-2a0e5466-551d-4a84-a1f3-d17bcc59a709'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.501484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] The token cache is being invalidate because project c487ee37e54042d680eccb72b046bb55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:20:58.501774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.502429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.503979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1207/2406] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:20:58 2022] DELETE /identity/v3/projects/c487ee37e54042d680eccb72b046bb55 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:20:58.508052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:58.509461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:58.523700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4385dd-5e26-421a-bbd9-e376685b22eb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:58.556101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:58.587841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:58.588732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:58.589022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:58.589306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] PATH_INFO: `/v3/users/fd20974ac5b44d9c9ea3e5bad4d3189d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:58.590644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd20974ac5b44d9c9ea3e5bad4d3189d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:58.593260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:58.651411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd20974ac5b44d9c9ea3e5bad4d3189d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.664376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd20974ac5b44d9c9ea3e5bad4d3189d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.675552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd20974ac5b44d9c9ea3e5bad4d3189d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.687215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd20974ac5b44d9c9ea3e5bad4d3189d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.687945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd20974ac5b44d9c9ea3e5bad4d3189d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:58.694885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1200/2407] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:20:58 2022] DELETE /identity/v3/users/fd20974ac5b44d9c9ea3e5bad4d3189d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:20:58.941315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:20:58.942832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:20:58.955698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5466-551d-4a84-a1f3-d17bcc59a709 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:20:58.989897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:20:59.015383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:20:59.016250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:20:59.016541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:20:59.016816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] PATH_INFO: `/v3/projects/30f6ed3c34b04720ba0587134fa1b300` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:20:59.018016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f6ed3c34b04720ba0587134fa1b300)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:20:59.019946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:20:59.087506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f6ed3c34b04720ba0587134fa1b300', 'request_id': 'req-0751f2da-b207-4885-a066-23dfa2021e40'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:59.096645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f6ed3c34b04720ba0587134fa1b300', 'request_id': 'req-0751f2da-b207-4885-a066-23dfa2021e40'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:59.108401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f6ed3c34b04720ba0587134fa1b300', 'request_id': 'req-0751f2da-b207-4885-a066-23dfa2021e40'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:59.124270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] The token cache is being invalidate because project 30f6ed3c34b04720ba0587134fa1b300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:20:59.124699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:59.125528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0751f2da-b207-4885-a066-23dfa2021e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:20:59.127309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1208/2408] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:20:58 2022] DELETE /identity/v3/projects/30f6ed3c34b04720ba0587134fa1b300 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:21:08.377279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:08.379073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:08.457073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabd7b0-e340-4518-a498-a10f1cc2adc8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:08.503432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:08.522856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:08.523551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:08.523832 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:08.524122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:08.525955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:08.538543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:08.540148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:08.550952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1201/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:21:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:21:09.119512 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:18.546438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=11 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:32.556547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f066d-d5b7-441a-a202-8448531383d3 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:32.556926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f066d-d5b7-441a-a202-8448531383d3 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:32.557194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f066d-d5b7-441a-a202-8448531383d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:32.580468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b24f066d-d5b7-441a-a202-8448531383d3 None None] MFA Rules not processed for user `34ec54486c9d404896f832b87f99bc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:32.581947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b24f066d-d5b7-441a-a202-8448531383d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:32.666451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1209/2410] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:21:39.814262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:39.816548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:39.897740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f33fe-5fd6-4c8c-98dd-3f294bc0992a service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:39.901793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:39.902703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:39.903084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:39.903477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:39.905602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:39.987604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:39.989217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:40.001621 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1202/2411] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:21:42.658618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=25 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:44.476459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:44.477092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:44.477092 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:44.498342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] MFA Rules not processed for user `e43d573fcf384f65b7382f2d0a01ade4`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:44.499604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:44.571962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1210/2412] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:21:45.145596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:45.147872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:45.173734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08e75-c9fa-4e92-833d-feb1e80e02aa service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:45.179645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:45.180591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.180872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.181147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] PATH_INFO: `/v3/users/dca54bc220104e79aa727f30399f3411` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.182517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca54bc220104e79aa727f30399f3411)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:45.184685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:45.240120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca54bc220104e79aa727f30399f3411'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.253838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca54bc220104e79aa727f30399f3411'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.269655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca54bc220104e79aa727f30399f3411'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.284701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca54bc220104e79aa727f30399f3411'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.285499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-c3e0daec-649e-4462-a3e9-f31c4fdf34af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca54bc220104e79aa727f30399f3411'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.293104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1203/2413] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:21:45 2022] DELETE /identity/v3/users/dca54bc220104e79aa727f30399f3411 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:21:45.490050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:45.520759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c856eb-ef1d-407a-b14d-b710eab3a317 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:45.602202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:45.603137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.603428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.603682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] PATH_INFO: `/v3/projects/7ea328a1154a4a9a955e411f7eab0aa9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.604877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea328a1154a4a9a955e411f7eab0aa9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:45.607053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:45.669990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea328a1154a4a9a955e411f7eab0aa9', 'request_id': 'req-fc4707e9-3f75-4521-b77c-109efad82814'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.676861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea328a1154a4a9a955e411f7eab0aa9', 'request_id': 'req-fc4707e9-3f75-4521-b77c-109efad82814'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.686615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea328a1154a4a9a955e411f7eab0aa9', 'request_id': 'req-fc4707e9-3f75-4521-b77c-109efad82814'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.698793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] The token cache is being invalidate because project 7ea328a1154a4a9a955e411f7eab0aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:21:45.699361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.700178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fc4707e9-3f75-4521-b77c-109efad82814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:21:45.702414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1211/2414] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:21:45 2022] DELETE /identity/v3/projects/7ea328a1154a4a9a955e411f7eab0aa9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:21:45.708204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.708575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.708801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.729385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:45.730921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:45.750043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:45.797364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1204/2415] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:21:45.805833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.806152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.806489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.825798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:45.827020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:45.835611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1212/2416] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:21:45.840433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:45.854210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3c2dc-c441-4771-ad08-c164b17db138 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:45.860446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:45.861115 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.861398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.861678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.862435 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:21:45.862747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:45.864355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:45.878816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1205/2417] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:21:45.884181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:45.895496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66addd-d202-4d54-b09d-a6e77ec3992d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:45.901328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:45.902933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:45.903249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:45.903497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:45.904330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:45.907148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:45.942445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:45.950714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1213/2418] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:21:45.954913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:45.965557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7c38-5ce6-465f-8b20-e9447bf67538 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:46.034234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:46.034946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.035223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.035521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.036212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:46.038156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:46.064483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1206/2419] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:21:45 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:21:46.074170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:46.089117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5993-4061-43e7-9ff7-e496584837a2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:46.095546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:46.096230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.096509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.096783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.097546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:46.099260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:46.110806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1214/2420] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:21:46.117266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:46.129339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ef804-cd2f-4d92-96ae-46032fe80e67 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:46.134810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:46.135492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.135776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.136058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.137155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:46.138805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0148c-aa07-4b2f-8bf3-9f2e8e3ebb14 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:46.149671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1207/2421] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:21:46.154235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:46.165074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8a49-e716-4c58-b386-7bcf2d5936f0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:46.172158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:46.173054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.173338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.173612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] PATH_INFO: `/v3/projects/0bd3f8996b5b46f6904e32023b092e5d/users/ccf66c5198f24f7cbc39132635c03611/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.190182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd3f8996b5b46f6904e32023b092e5d, user_id=ccf66c5198f24f7cbc39132635c03611, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:46.192940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:46.208633 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1215/2422] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:21:46 2022] PUT /identity/v3/projects/0bd3f8996b5b46f6904e32023b092e5d/users/ccf66c5198f24f7cbc39132635c03611/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:21:46.214143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74083-3407-4236-999d-83d0b658c7c5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.214471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74083-3407-4236-999d-83d0b658c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.214753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74083-3407-4236-999d-83d0b658c7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.233971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4af74083-3407-4236-999d-83d0b658c7c5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:46.235330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4af74083-3407-4236-999d-83d0b658c7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:46.297672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1208/2423] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:21:46.307917 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:46.310008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:46.383431 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a894917-aff8-4b39-acf4-1683d537f362 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:46.388155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:46.388839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:46.389106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:46.389383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:46.400781 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:46.402350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:46.415176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1216/2424] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:21:51.073267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbab3f-86cd-4308-832b-23f6fc18c26f None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:51.073745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbab3f-86cd-4308-832b-23f6fc18c26f None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:51.073843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbab3f-86cd-4308-832b-23f6fc18c26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:51.102499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d2cbab3f-86cd-4308-832b-23f6fc18c26f None None] MFA Rules not processed for user `ccf66c5198f24f7cbc39132635c03611`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:51.103642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d2cbab3f-86cd-4308-832b-23f6fc18c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:51.120662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:21:51.170835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1209/2425] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:21:51.186120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:21:51.187653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:51.273817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f29ce-454d-45a7-adfc-98cdaf3ebe48 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:21:51.279400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:21:51.280071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:51.280355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:51.280639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:51.289162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:21:51.290776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:21:51.299912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1217/2426] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:21:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:21:52.894116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554f3f-93f5-40ad-b057-a349bfd94c05 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:21:52.894975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554f3f-93f5-40ad-b057-a349bfd94c05 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:21:52.895528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554f3f-93f5-40ad-b057-a349bfd94c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:21:52.920150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-da554f3f-93f5-40ad-b057-a349bfd94c05 None None] MFA Rules not processed for user `ccf66c5198f24f7cbc39132635c03611`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:21:52.921302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-da554f3f-93f5-40ad-b057-a349bfd94c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:21:52.929287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1210/2427] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:22:01.296079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:22:02.912906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=22 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:22:44.749687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:22:44.751380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:22:44.827997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24520a94-72c4-411c-892f-7a879b29528b service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:22:44.833291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:22:44.833916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:22:44.834195 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:22:44.834500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:22:44.836320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:22:44.848509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:22:44.850115 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:22:44.860123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1218/2428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:22:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:22:45.417525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0164dd-8952-41de-a9e4-20b6963cd660 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:22:45.418054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0164dd-8952-41de-a9e4-20b6963cd660 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:22:45.418158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0164dd-8952-41de-a9e4-20b6963cd660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:22:45.436225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6d0164dd-8952-41de-a9e4-20b6963cd660 None None] MFA Rules not processed for user `4041faa2919d4760ae10907070a10dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:22:45.437426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6d0164dd-8952-41de-a9e4-20b6963cd660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:22:45.493713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1211/2429] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:22:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:22:54.856034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=32 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:22:55.486193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=25 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:23:17.808920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:23:17.811295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:23:17.830927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23949d6-3042-4d5d-a8b2-d60799eaee7c service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:23:17.837626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:23:17.838279 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:23:17.838596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:23:17.838876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:23:17.840975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:23:17.900599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:23:17.902149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:23:17.911505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1219/2430] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:23:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:23:27.907189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=28 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:23:44.005800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7ff6-835a-429a-ae99-cd091e770de2 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:23:44.006349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7ff6-835a-429a-ae99-cd091e770de2 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:23:44.006349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c7ff6-835a-429a-ae99-cd091e770de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:23:44.028120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-886c7ff6-835a-429a-ae99-cd091e770de2 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:23:44.029429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-886c7ff6-835a-429a-ae99-cd091e770de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:23:44.090873 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1212/2431] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:23:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:23:44.142938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:23:44.144475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:23:44.228844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595c4e-3c93-4feb-9e94-1208865c9622 service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:23:44.234872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:23:44.235572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:23:44.236315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:23:44.236315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:23:44.247112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:23:44.248733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:23:44.258962 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1220/2432] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:23:54.083247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=25 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:23:54.254245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=32 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:24:00.462247 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:00.462751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:00.462866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:00.483179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:24:00.484389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:00.492868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1213/2433] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:24:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:24:00.505109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:00.517028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89195-7f69-4439-88a9-be1c06ac9da3 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:00.524044 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:00.524700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:00.525009 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:00.525340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:00.534952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:00.536605 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165befad-64b0-4946-989a-ede3ee6613aa service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:00.544787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1221/2434] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:24:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:24:00.933477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:00.934932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:01.000865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b0b0-b575-428c-a34c-62c54d1816cf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:01.006215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:01.006215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:01.006215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:01.006215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:01.009103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:01.111513 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:01.112906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82019-00e8-423a-a520-fc91258ebe4d service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:01.127339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1214/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:24:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:24:10.541419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:24:11.119600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=51 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:24:15.343523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:15.344125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:15.344125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:15.371090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] MFA Rules not processed for user `34ec54486c9d404896f832b87f99bc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:24:15.372906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:15.381475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1222/2436] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:24:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:24:24.602043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:24.602587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:24.602684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:24.623648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:24:24.625210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:24.637517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1215/2437] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:24:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:24:24.651900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:24.664895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf31e-69f0-40fb-b328-52e2620e7337 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:24.671356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:24.672131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:24.672415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:24.672715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:24.681519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:24.683130 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:24.691961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1223/2438] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:24:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:24:31.772700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:31.774404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:31.787074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ee18b-50e5-432c-aa84-f5a0fe724192 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:31.791279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:31.791944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:31.792214 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:31.792489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:31.794454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:31.808173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:31.809669 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:31.821070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1216/2439] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:24:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:24:33.008980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:33.019331 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02269ea2-2f79-483d-ad05-d6529e054258 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:33.023761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:33.024400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:33.024681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:33.024955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:33.026991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:33.040954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:33.042417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:33.053556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1224/2440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:24:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:24:34.540454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:34.552627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b53-7cad-44a8-bd87-78c357352f45 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:34.556704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:34.557341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:34.557622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:34.557898 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:34.559947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:34.629962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:34.631325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:34.642642 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1217/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:24:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:24:43.048774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=15 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:24:44.637531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=39 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:24:58.149471 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:24:58.158661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637560-91f8-4d73-afa5-4a65781dffee service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:24:58.163256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:24:58.163896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:24:58.164262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:24:58.164533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:24:58.166714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:24:58.180051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:24:58.181476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:24:58.191538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1225/2442] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:01.386924 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:01.388732 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:01.404784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61647bef-763d-4c55-aed5-bde122161d95 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:01.410405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:01.411042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:01.411324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:01.411600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:01.413542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:01.426679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:01.428639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:01.441741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:01.443395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:01.450708 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1218/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:01.458269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f8a48-468b-45f9-b46b-cf61ff445c59 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:01.463284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:01.464231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:01.464510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:01.464897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] PATH_INFO: `/v3/users/4041faa2919d4760ae10907070a10dc7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:01.480778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4041faa2919d4760ae10907070a10dc7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:01.482747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:01.531472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4041faa2919d4760ae10907070a10dc7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.540576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4041faa2919d4760ae10907070a10dc7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.541216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4041faa2919d4760ae10907070a10dc7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.548390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4041faa2919d4760ae10907070a10dc7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.560409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3bd83f43-8f7c-463a-8cf8-f9b865603edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4041faa2919d4760ae10907070a10dc7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.566816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1226/2444] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:01 2022] DELETE /identity/v3/users/4041faa2919d4760ae10907070a10dc7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:25:01.760411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:01.773258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39f88-3123-4154-a725-cdf487a29d49 service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:01.832521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:01.833333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:01.833618 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:01.833897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] PATH_INFO: `/v3/projects/7ccd298eebe6413daaae0d801e902616` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:01.844845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ccd298eebe6413daaae0d801e902616)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:01.846868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:01.900238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ccd298eebe6413daaae0d801e902616', 'request_id': 'req-507200e5-56ab-4686-a194-5f8246346cd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.909730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ccd298eebe6413daaae0d801e902616', 'request_id': 'req-507200e5-56ab-4686-a194-5f8246346cd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.919221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ccd298eebe6413daaae0d801e902616', 'request_id': 'req-507200e5-56ab-4686-a194-5f8246346cd3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.936421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] The token cache is being invalidate because project 7ccd298eebe6413daaae0d801e902616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:01.936729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.937324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-507200e5-56ab-4686-a194-5f8246346cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:01.938925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1219/2445] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:01 2022] DELETE /identity/v3/projects/7ccd298eebe6413daaae0d801e902616 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:01.944191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:01.944191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:01.944758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:01.963822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:01.965068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:02.032442 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1227/2446] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:02.040879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.041176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.041458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.061475 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:02.062760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:02.072074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1220/2447] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:02.076935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.090587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffdc8-7b16-421e-b1e6-cc26b5ef15b6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.096118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.096811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.097089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.097363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.098104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:25:02.098402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.100047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.117685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1228/2448] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:25:02.123869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.139360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb449905-29f0-40b5-8f1e-03bbcfd9d8f1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.148058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.148930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.149218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.149499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.150266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.152223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.182840 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1221/2449] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:02.186984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.197456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94442307-d502-4079-a522-cc3d89735684 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.231345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:02.265999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.266708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.267011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.267282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.267974 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.269889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.290116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1229/2450] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:02.294406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.305173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb119b-8456-4b00-abf3-efca5176dcd8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.311027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.311694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.311949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.312223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.312956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.314637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.325436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1222/2451] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:25:02.329570 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.340742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e20d1-e8dc-41b3-85d9-76a9bc05c45f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.348818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.349756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.350067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.350359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.351419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.353152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540f53-0184-485a-b787-8a6e79b0e22e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.363711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1230/2452] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:02.367987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.378323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3203298-20e6-446e-b9b5-4669bf4d230b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.383458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.384378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.385001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.385272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] PATH_INFO: `/v3/projects/b5de86b9e81543fb94ef7262645306dc/users/1111abeb885043c592fa24cc04515dee/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.400552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5de86b9e81543fb94ef7262645306dc, user_id=1111abeb885043c592fa24cc04515dee, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.403245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.418616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1223/2453] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:02 2022] PUT /identity/v3/projects/b5de86b9e81543fb94ef7262645306dc/users/1111abeb885043c592fa24cc04515dee/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:02.429723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.430020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.430289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.458307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] MFA Rules not processed for user `1111abeb885043c592fa24cc04515dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:02.459963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:02.531071 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1231/2454] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:02.536401 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.542782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:02.604894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad9d90-46da-4dcb-806c-adeb27532cee admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.609908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.611275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.611571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.611851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.612568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.614507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.644762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1224/2455] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:02.649178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.664726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e57916-122b-484c-a8d6-de0a5b67911d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.728615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.729317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.729607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.729891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.730700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.732560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.755378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1232/2456] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:02.759841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.770655 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd3311-2305-43fc-8eb7-edf69150f57e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.775866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.776545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.776834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.777116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.777851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.779503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.791030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1225/2457] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:02.795227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.805976 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a949096-0178-47c2-9901-2f7ce5bb5c5d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.810989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.811807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.812090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.812337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] PATH_INFO: `/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b/users/1232720d6c87427b8985aca9a035fc80/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.829658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383f07b01a3a4657b3ad2a42ac1f522b, user_id=1232720d6c87427b8985aca9a035fc80, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.832455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.847649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1233/2458] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:02 2022] PUT /identity/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b/users/1232720d6c87427b8985aca9a035fc80/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:25:02.851955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.861920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7e19-334b-4283-af23-3cd59ac43bb2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.919573 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.920244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.920525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.920799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.921525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.923178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26998-4708-44d0-b7c2-806d3172a4b2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.933133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1226/2459] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:25:02.937429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:02.948069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0681e-de2c-4b10-88f7-ea8a27588e61 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:02.953391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:02.954300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.954597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.954868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] PATH_INFO: `/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b/users/1232720d6c87427b8985aca9a035fc80/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:02.957050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383f07b01a3a4657b3ad2a42ac1f522b, user_id=1232720d6c87427b8985aca9a035fc80, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:02.959739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:02.963103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:02.972913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1234/2460] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:02 2022] PUT /identity/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b/users/1232720d6c87427b8985aca9a035fc80/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:02.983360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:02.983656 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:02.983927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:03.020158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] MFA Rules not processed for user `1232720d6c87427b8985aca9a035fc80`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:03.022015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:03.097634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:03.101909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1227/2461] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:03.114112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:03.116553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:03.181189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd48b7a-28fe-4f0f-9b80-7de7d8291521 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:03.185640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:03.186284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:03.186615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:03.186892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:03.196946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:03.198482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:03.208308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1235/2462] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:03.283781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:03.295307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9514ed-1012-45dd-8381-2584658f6ce8 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:03.299718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:03.300391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:03.300662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:03.300940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:03.360357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:03.361713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3abe45-8fe5-403c-971c-d84bd25c0c0f service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:03.372136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1228/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:13.203847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=28 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:13.366977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=28 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:14.421608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:14.484010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778bde7-ac3a-400d-8acf-9318bb00bcbc service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:14.488953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:14.489829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:14.490114 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:14.490407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] PATH_INFO: `/v3/users/1111abeb885043c592fa24cc04515dee` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:14.491664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1111abeb885043c592fa24cc04515dee)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:14.493585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:14.544698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1111abeb885043c592fa24cc04515dee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:14.552831 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1111abeb885043c592fa24cc04515dee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:14.553566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1111abeb885043c592fa24cc04515dee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:14.561065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1111abeb885043c592fa24cc04515dee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:14.573830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1111abeb885043c592fa24cc04515dee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:14.580372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1236/2464] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:14 2022] DELETE /identity/v3/users/1111abeb885043c592fa24cc04515dee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:14.584998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:14.585497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:14.585784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:14.605319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:14.606775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:14.674596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1229/2465] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:25:14.685035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:14.686593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:14.745843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475df54-9328-4173-aaa3-ee3597f1c658 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:14.750036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:14.750645 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:14.750926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:14.751222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:14.761194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:14.762686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:14.772412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1237/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:14.930650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:14.941969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64835e-30e7-44a8-8522-7be94599b326 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:14.959780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:14.961161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:14.961418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:14.961736 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:14.963663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:14.975252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:14.976740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:14.991278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1230/2467] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:15.181132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:15.183172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:15.201865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46534a2-73ca-4017-bd20-e755608faaaa service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:15.208520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:15.209757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:15.210100 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:15.210539 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] PATH_INFO: `/v3/projects/b5de86b9e81543fb94ef7262645306dc` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:15.212619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5de86b9e81543fb94ef7262645306dc)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:15.216615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:15.239396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:15.275483 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5de86b9e81543fb94ef7262645306dc', 'request_id': 'req-564c1939-cb46-4885-98fc-d0679cb8ee57'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.282380 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5de86b9e81543fb94ef7262645306dc', 'request_id': 'req-564c1939-cb46-4885-98fc-d0679cb8ee57'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.291893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5de86b9e81543fb94ef7262645306dc', 'request_id': 'req-564c1939-cb46-4885-98fc-d0679cb8ee57'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.303457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] The token cache is being invalidate because project b5de86b9e81543fb94ef7262645306dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:15.303727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.304403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.306222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1238/2468] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:15 2022] DELETE /identity/v3/projects/b5de86b9e81543fb94ef7262645306dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:15.310312 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:15.322515 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca374309-e013-46e9-8a5a-654ae55890b1 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:15.382184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:15.383110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:15.383393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:15.383674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] PATH_INFO: `/v3/users/1232720d6c87427b8985aca9a035fc80` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:15.385268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1232720d6c87427b8985aca9a035fc80)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:15.387462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:15.436842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1232720d6c87427b8985aca9a035fc80'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.449056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1232720d6c87427b8985aca9a035fc80'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.457893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1232720d6c87427b8985aca9a035fc80'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.466506 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1232720d6c87427b8985aca9a035fc80'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.467178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-10b7d6d7-5933-429a-b869-5e69e6d8bd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1232720d6c87427b8985aca9a035fc80'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.473954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1231/2469] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:15 2022] DELETE /identity/v3/users/1232720d6c87427b8985aca9a035fc80 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:15.763761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:15.765324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:15.780062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c1939-cb46-4885-98fc-d0679cb8ee57 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:15.841306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:15.842202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:15.842620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:15.842906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] PATH_INFO: `/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:15.844151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] RBAC: Authorizing `identity:delete_project(project_id=383f07b01a3a4657b3ad2a42ac1f522b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:15.846060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:15.896488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383f07b01a3a4657b3ad2a42ac1f522b', 'request_id': 'req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.903251 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383f07b01a3a4657b3ad2a42ac1f522b', 'request_id': 'req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.912040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383f07b01a3a4657b3ad2a42ac1f522b', 'request_id': 'req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.923087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] The token cache is being invalidate because project 383f07b01a3a4657b3ad2a42ac1f522b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:15.923372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.924004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7ccf20d1-72af-4c10-af4a-ac1e6a97263f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:15.925578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1239/2470] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:15 2022] DELETE /identity/v3/projects/383f07b01a3a4657b3ad2a42ac1f522b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:15.930630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:15.931128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:15.931234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:15.943757 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:15.950315 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:15.952751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:16.018032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1232/2471] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:16.026989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.027273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.027662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.046722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:16.048022 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:16.056566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1240/2472] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:16.061014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.070861 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6988-b4c4-497c-913f-a0e8e561bdbf None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.075806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.076467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.076755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.077029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.077752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:25:16.078060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.079747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.091887 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1233/2473] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:25:16.096194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.106123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9677-9a1c-412a-afb6-b0b87bbb41b8 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.111188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.111858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.112142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.112417 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.113120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.115070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.142737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1241/2474] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:16.146842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.156776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b54163-e9d6-4e5e-b7bb-06acf142a738 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.221269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.221941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.222191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.222512 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.223204 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.225130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.247939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1234/2475] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/users => generated 278 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:16.252441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.267813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326cd46-e5cf-4828-89ae-61d43bf1d3fb admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.274266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.274952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.275232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.275506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.276241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.279043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.280887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:16.295789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1242/2476] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:16.300895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.314079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770340-9159-4351-b9c3-abf3c48d32d3 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.319795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.320458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.320737 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.321009 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.321735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.323398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc79cda-2e2e-4b27-a333-60dc4d8c7f26 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.334797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1235/2477] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:16.339106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.351562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b7b45-f852-4dfb-bf77-5655d2ca143b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.360370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.361612 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.361927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.362305 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] PATH_INFO: `/v3/projects/31257255b7b341c8b0a005aa80b0f419/users/cfa384c7511f4ad785bef8c9cbf12246/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.381222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31257255b7b341c8b0a005aa80b0f419, user_id=cfa384c7511f4ad785bef8c9cbf12246, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.383995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.399258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1243/2478] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:16 2022] PUT /identity/v3/projects/31257255b7b341c8b0a005aa80b0f419/users/cfa384c7511f4ad785bef8c9cbf12246/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:16.409686 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.409979 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.410262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.437993 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] MFA Rules not processed for user `cfa384c7511f4ad785bef8c9cbf12246`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:16.439238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:16.505961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1236/2479] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:16.511213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.573388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637db547-3522-4d01-ad22-872015037e97 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.578656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.579304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.579591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.579870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.580561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.582482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.614020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1244/2480] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:16.618470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.625047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:16.633164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3efe71-8c0c-4709-a4c7-a409cd5aed6c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.698885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.699645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.699964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.700244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.700966 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.702949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.723892 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1237/2481] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:16.727908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.738850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49545712-5f40-4caf-bc30-c3b15fd04b3e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.744052 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.744725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.745015 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.745345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.746180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.747846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.759025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1245/2482] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:16.763517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.774565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9590c-5296-4708-abc7-9a7734137628 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.780697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.781612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.781885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.782162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] PATH_INFO: `/v3/projects/7e9095f1a91644b99605b44979a6f8ed/users/37eb7ff688db4aadbe6916c0500a927a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.799299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9095f1a91644b99605b44979a6f8ed, user_id=37eb7ff688db4aadbe6916c0500a927a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.801952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.816548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1238/2483] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:16 2022] PUT /identity/v3/projects/7e9095f1a91644b99605b44979a6f8ed/users/37eb7ff688db4aadbe6916c0500a927a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:16.820551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.831065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc508-8bad-4c4b-a2f7-c6c0aa8acae5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.891714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.892368 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.892665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.892943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.893698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.895355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6132f-4949-40c7-b811-dde191a2a37b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.905228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1246/2484] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:25:16.909391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:16.919754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc1bb3-bf02-4ba2-8f79-edd5a319ffdb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:16.924672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:16.925553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.925845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.926108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] PATH_INFO: `/v3/projects/7e9095f1a91644b99605b44979a6f8ed/users/37eb7ff688db4aadbe6916c0500a927a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.928161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9095f1a91644b99605b44979a6f8ed, user_id=37eb7ff688db4aadbe6916c0500a927a, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:16.930823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:16.944706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1239/2485] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:16 2022] PUT /identity/v3/projects/7e9095f1a91644b99605b44979a6f8ed/users/37eb7ff688db4aadbe6916c0500a927a/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:16.954451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:16.954744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:16.955017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:16.982975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] MFA Rules not processed for user `37eb7ff688db4aadbe6916c0500a927a`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:16.984198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:17.016008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:17.056366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1247/2486] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:16 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:17.068416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:17.071439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:17.136415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bf7c-f598-435e-9e6a-0e6caeaece6a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:17.140943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:17.141569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:17.141849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:17.142121 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:17.152325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:17.153768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:17.163902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1240/2487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:17 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:17.239393 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:17.249699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a7f6d-8a55-4b4f-868e-009dd4c1f395 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:17.253750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:17.254444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:17.254730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:17.255007 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:17.315246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:17.316675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac07743-3c34-49da-af80-93f4d7fc8f74 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:17.329038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1248/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:17 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:27.159416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:27.324843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=43 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:32.527862 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:32.593130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb90dc-dfdb-4a51-bfee-22311f412a1b service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:32.598721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:32.599575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:32.599876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:32.600375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] PATH_INFO: `/v3/users/cfa384c7511f4ad785bef8c9cbf12246` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:32.601680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa384c7511f4ad785bef8c9cbf12246)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:32.603639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:32.653714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa384c7511f4ad785bef8c9cbf12246'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:32.665462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa384c7511f4ad785bef8c9cbf12246'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:32.674439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa384c7511f4ad785bef8c9cbf12246'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:32.682039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa384c7511f4ad785bef8c9cbf12246'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:32.682712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa384c7511f4ad785bef8c9cbf12246'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:32.688602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1241/2489] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:32 2022] DELETE /identity/v3/users/cfa384c7511f4ad785bef8c9cbf12246 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:32.693977 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:32.694490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:32.694615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:32.718895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:32.721227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:32.787314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1249/2490] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:32.801822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:32.803646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:32.865703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967199b-b75d-46a8-9c1d-1d2a1d12e91a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:32.870836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:32.871492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:32.871811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:32.872091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:32.882014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:32.883488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:32.897136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1242/2491] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:33.178688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:33.180169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:33.192856 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff5ed1-5109-4d04-9c85-cba056f25342 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:33.199152 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:33.200064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:33.200349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:33.200622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] PATH_INFO: `/v3/projects/31257255b7b341c8b0a005aa80b0f419` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:33.201927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31257255b7b341c8b0a005aa80b0f419)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:33.203911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:33.252677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31257255b7b341c8b0a005aa80b0f419', 'request_id': 'req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.259481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31257255b7b341c8b0a005aa80b0f419', 'request_id': 'req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.268648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31257255b7b341c8b0a005aa80b0f419', 'request_id': 'req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.279089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] The token cache is being invalidate because project 31257255b7b341c8b0a005aa80b0f419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:33.279401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.280041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.281634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1250/2492] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:33 2022] DELETE /identity/v3/projects/31257255b7b341c8b0a005aa80b0f419 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:33.285690 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:33.295730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9040831-4835-4788-b826-fab103ebad03 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:33.314810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:33.356246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:33.357162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:33.357442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:33.357739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] PATH_INFO: `/v3/users/37eb7ff688db4aadbe6916c0500a927a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:33.359073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37eb7ff688db4aadbe6916c0500a927a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:33.361046 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:33.416191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37eb7ff688db4aadbe6916c0500a927a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.429194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37eb7ff688db4aadbe6916c0500a927a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.437349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37eb7ff688db4aadbe6916c0500a927a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.444345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37eb7ff688db4aadbe6916c0500a927a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.444976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5b1e124a-4b8f-4339-afae-bd9d75ecbc06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37eb7ff688db4aadbe6916c0500a927a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.451210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1243/2493] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:33 2022] DELETE /identity/v3/users/37eb7ff688db4aadbe6916c0500a927a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:33.740132 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:33.741696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:33.755318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a034a-1a3b-4566-bac0-07f8ab9b8062 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:33.817093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:33.817876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:33.818160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:33.818484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] PATH_INFO: `/v3/projects/7e9095f1a91644b99605b44979a6f8ed` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:33.819658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9095f1a91644b99605b44979a6f8ed)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:33.821561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:33.856255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:33.874287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9095f1a91644b99605b44979a6f8ed', 'request_id': 'req-cab1f014-eb42-4041-8264-2b737d5ced24'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.880708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9095f1a91644b99605b44979a6f8ed', 'request_id': 'req-cab1f014-eb42-4041-8264-2b737d5ced24'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.891416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9095f1a91644b99605b44979a6f8ed', 'request_id': 'req-cab1f014-eb42-4041-8264-2b737d5ced24'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.902451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] The token cache is being invalidate because project 7e9095f1a91644b99605b44979a6f8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:33.902740 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.903376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cab1f014-eb42-4041-8264-2b737d5ced24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:33.904956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1251/2494] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:33 2022] DELETE /identity/v3/projects/7e9095f1a91644b99605b44979a6f8ed => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:33.910142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:33.910652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:33.910746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:33.928769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:33.929983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:33.995111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1244/2495] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:34.004830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.004830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.004830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.031129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:34.032377 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:34.042158 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1252/2496] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:34.046707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.057128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4a65-fe69-4be5-aa70-f172faaac7fd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.062466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.063175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.063465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.063744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.064485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:25:34.064800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.066507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.084008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1245/2497] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:25:34.085260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.095382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f760ac2-5d02-4d96-b824-9a70a3f39227 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.100462 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.101219 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.101507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.101789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.102524 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.104435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.132284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1253/2498] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:34.136652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.147381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdd3ca-3f9a-4c9e-a931-960915abf44a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.175331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:34.207493 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.208168 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.208454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.208731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.209429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.211393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.233309 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1246/2499] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:34.237496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.248676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2be1ec-00cd-4ab3-900b-47a962a381ea admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.254562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.255226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.255514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.255798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.256533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.258143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.269326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1254/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:34.274390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.285040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fc99-bb89-4c24-bcb1-c42f1b8c2e2b admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.290205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.290903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.291188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.291465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.292200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.294016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15d2c8-8ad1-4940-a9d5-5fc4a936caf3 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.304432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1247/2501] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:34.308887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.319258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a2ff5-bd0c-4729-b94c-bd9d405ac56f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.324946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.325830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.326112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.326411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] PATH_INFO: `/v3/projects/896aedbfe0f246d899f3e5c57bfe857a/users/565d8bd9d2574d658b78e50709bb810e/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.341238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=896aedbfe0f246d899f3e5c57bfe857a, user_id=565d8bd9d2574d658b78e50709bb810e, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.343810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.360752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1255/2502] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:34 2022] PUT /identity/v3/projects/896aedbfe0f246d899f3e5c57bfe857a/users/565d8bd9d2574d658b78e50709bb810e/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:34.371152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.371602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.372133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.400734 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] MFA Rules not processed for user `565d8bd9d2574d658b78e50709bb810e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:34.401956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:34.467793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1248/2503] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:34.472541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.540507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9dda4-cb1f-4c75-9769-07dd6e132d6b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.546464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.547145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.547433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.547712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.548419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.550348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.578722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1256/2504] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:34.582877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.594077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e51cd2-7fdd-4198-b839-02c9a0abdf16 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.653990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.654711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.654998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.655272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.656004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.657935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.680335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1249/2505] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:34.685671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.696968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fe69d-aded-48c9-a0d0-072a8f5fa47b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.702429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.703104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.703390 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.703664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.704392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.705991 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.716855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1257/2506] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:34.721160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.731461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d03e-483c-448f-95d6-1266f4cea56f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.736950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.737610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.737863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.738155 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.738921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.740536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ad988-daa3-43b0-819d-2cef8584fe25 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.750780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1250/2507] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:34.755103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.765910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef7062-941b-4a1e-8db6-0781c03b9340 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:34.771014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:34.771895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.772178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.772451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] PATH_INFO: `/v3/projects/2f1c687ef0ea457db5dc7ed6901c9631/users/d9bb85c033be4edc85d9bf8ced9be72c/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.787429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1c687ef0ea457db5dc7ed6901c9631, user_id=d9bb85c033be4edc85d9bf8ced9be72c, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:34.790062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:34.800362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:34.805176 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1258/2508] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:34 2022] PUT /identity/v3/projects/2f1c687ef0ea457db5dc7ed6901c9631/users/d9bb85c033be4edc85d9bf8ced9be72c/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:34.816521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:34.816814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:34.817096 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:34.845663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] MFA Rules not processed for user `d9bb85c033be4edc85d9bf8ced9be72c`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:34.846852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:34.851085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:34.924847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1251/2509] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:34.929521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:34.996507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9c9b-5dab-4a60-92d4-25bf81536da1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.002474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.003143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.003435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.003710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.004411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.006327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.041492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1259/2510] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:34 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:35.047101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.058334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737928a8-6ce2-4804-a7da-c93501cf1c5b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.116044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.116640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.116917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.117190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.117877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.119833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.140897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1252/2511] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:35 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:35.144946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.155478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b41ce-72e8-4384-bdec-3c751de24534 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.160900 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.161565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.161844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.162118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.162884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.164475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.175413 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1260/2512] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:35.179492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.190144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc4816-c665-44b2-a306-14bff10efae4 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.195143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.196034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.196313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.196653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] PATH_INFO: `/v3/projects/52a6e21a32514f7086f8887dadacc9d9/users/39d3f7cc8081419d8ef82d0f6081348d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.211755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a6e21a32514f7086f8887dadacc9d9, user_id=39d3f7cc8081419d8ef82d0f6081348d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.214451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.232477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1253/2513] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:35 2022] PUT /identity/v3/projects/52a6e21a32514f7086f8887dadacc9d9/users/39d3f7cc8081419d8ef82d0f6081348d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:35.236843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.247288 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279931e-bc4f-4bd6-b464-fd60055d7d0f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.304853 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.305687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.305966 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.306236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.306990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.308551 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab07f-98f7-4af0-95c5-c2bccd917059 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.318667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1261/2514] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:35.323190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.334517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3bb0b-9cfa-4ba4-aed5-de4cca6f9cc2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.340125 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.341056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.341342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.341657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] PATH_INFO: `/v3/projects/52a6e21a32514f7086f8887dadacc9d9/users/39d3f7cc8081419d8ef82d0f6081348d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.343969 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a6e21a32514f7086f8887dadacc9d9, user_id=39d3f7cc8081419d8ef82d0f6081348d, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.346735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.357830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1254/2515] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:35 2022] PUT /identity/v3/projects/52a6e21a32514f7086f8887dadacc9d9/users/39d3f7cc8081419d8ef82d0f6081348d/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:35.367923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.368144 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.368420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.395124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] MFA Rules not processed for user `39d3f7cc8081419d8ef82d0f6081348d`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:35.396316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:35.458662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:35.471658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1262/2516] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:25:35.485482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.487319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:35.549008 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eddec-019b-4a2f-88b6-116112ccf125 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.553039 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.553697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.553978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.554256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.563803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.565236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.570175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:35.575218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1255/2517] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:35.615686 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:35.617125 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:35.629872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092853f7-0bf8-493d-aee1-0a15a63309db None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:35.633837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:35.634891 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:35.635128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:35.635545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:35.694819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:35.696292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a48968-d902-4e1c-a991-4ad9e1f3924f service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:35.706035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1263/2518] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:38.630658 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:38.692864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2112-db0d-402b-bcca-4f0e13c7606c service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:38.697660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:38.698581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:38.698868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:38.699146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] PATH_INFO: `/v3/users/565d8bd9d2574d658b78e50709bb810e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:38.700349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565d8bd9d2574d658b78e50709bb810e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:38.702281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:38.763231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565d8bd9d2574d658b78e50709bb810e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:38.778816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565d8bd9d2574d658b78e50709bb810e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:38.788094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565d8bd9d2574d658b78e50709bb810e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:38.795482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565d8bd9d2574d658b78e50709bb810e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:38.796129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565d8bd9d2574d658b78e50709bb810e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:38.802118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1256/2519] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:38 2022] DELETE /identity/v3/users/565d8bd9d2574d658b78e50709bb810e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:38.807406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:38.807955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:38.808054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:38.825446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:38.826705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:38.892571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1264/2520] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:38.902775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:38.904335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:38.962967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b5ca1-16c2-4588-8e83-34f8ac10f836 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:38.966875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:38.967535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:38.967816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:38.968090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:38.977304 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:38.978773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:38.988369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1257/2521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:39.291116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:39.292714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:39.305391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8259ca-7bb9-4f76-ba77-61162ac1c8ae None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:39.310538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:39.311496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:39.311768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:39.312051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] PATH_INFO: `/v3/projects/896aedbfe0f246d899f3e5c57bfe857a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:39.313444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] RBAC: Authorizing `identity:delete_project(project_id=896aedbfe0f246d899f3e5c57bfe857a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:39.315474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:39.366600 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896aedbfe0f246d899f3e5c57bfe857a', 'request_id': 'req-33543a11-ef27-4337-ad4e-098f46bb057c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.372793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896aedbfe0f246d899f3e5c57bfe857a', 'request_id': 'req-33543a11-ef27-4337-ad4e-098f46bb057c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.382243 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896aedbfe0f246d899f3e5c57bfe857a', 'request_id': 'req-33543a11-ef27-4337-ad4e-098f46bb057c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.393383 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] The token cache is being invalidate because project 896aedbfe0f246d899f3e5c57bfe857a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:39.393669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.394352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.395944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1265/2522] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:39 2022] DELETE /identity/v3/projects/896aedbfe0f246d899f3e5c57bfe857a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:39.400154 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:39.410054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c482-37b0-4273-b576-16dfcb8dcb79 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:39.428600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:39.469521 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:39.470403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:39.470721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:39.470999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] PATH_INFO: `/v3/users/d9bb85c033be4edc85d9bf8ced9be72c` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:39.472267 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bb85c033be4edc85d9bf8ced9be72c)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:39.474196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:39.527343 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bb85c033be4edc85d9bf8ced9be72c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.540629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bb85c033be4edc85d9bf8ced9be72c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.549026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bb85c033be4edc85d9bf8ced9be72c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.556101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bb85c033be4edc85d9bf8ced9be72c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.556740 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bb85c033be4edc85d9bf8ced9be72c'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:39.563075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1258/2523] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:39 2022] DELETE /identity/v3/users/d9bb85c033be4edc85d9bf8ced9be72c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:39.857650 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:39.859354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:39.872720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543a11-ef27-4337-ad4e-098f46bb057c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:39.887547 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:39.939425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:39.940313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:39.940640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:39.940998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] PATH_INFO: `/v3/projects/2f1c687ef0ea457db5dc7ed6901c9631` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:39.942798 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1c687ef0ea457db5dc7ed6901c9631)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:39.945771 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:40.004440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1c687ef0ea457db5dc7ed6901c9631', 'request_id': 'req-48d948bc-6872-4589-821c-eefd47b4c50c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.014520 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1c687ef0ea457db5dc7ed6901c9631', 'request_id': 'req-48d948bc-6872-4589-821c-eefd47b4c50c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.024703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1c687ef0ea457db5dc7ed6901c9631', 'request_id': 'req-48d948bc-6872-4589-821c-eefd47b4c50c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.036336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] The token cache is being invalidate because project 2f1c687ef0ea457db5dc7ed6901c9631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:40.036622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.037302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.038953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1266/2524] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:39 2022] DELETE /identity/v3/projects/2f1c687ef0ea457db5dc7ed6901c9631 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:40.043215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:40.044762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:40.059676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84775f50-36db-492d-b525-6ad5b7270b8e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:40.128060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:40.128956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.129238 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.129527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] PATH_INFO: `/v3/users/39d3f7cc8081419d8ef82d0f6081348d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.130780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d3f7cc8081419d8ef82d0f6081348d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:40.132728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:40.180976 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:40.184076 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d3f7cc8081419d8ef82d0f6081348d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.197741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d3f7cc8081419d8ef82d0f6081348d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.206675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d3f7cc8081419d8ef82d0f6081348d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.214021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d3f7cc8081419d8ef82d0f6081348d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.214685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0e6a9a74-dc97-4d7b-b3b0-9f5a739eb246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d3f7cc8081419d8ef82d0f6081348d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.220829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1259/2525] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:40 2022] DELETE /identity/v3/users/39d3f7cc8081419d8ef82d0f6081348d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:40.503091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:40.504618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:40.518258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d948bc-6872-4589-821c-eefd47b4c50c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:40.578907 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:40.579789 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.580067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.580338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] PATH_INFO: `/v3/projects/52a6e21a32514f7086f8887dadacc9d9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.581648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a6e21a32514f7086f8887dadacc9d9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:40.583628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:40.634602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a6e21a32514f7086f8887dadacc9d9', 'request_id': 'req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.635795 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:40.642889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a6e21a32514f7086f8887dadacc9d9', 'request_id': 'req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.652351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a6e21a32514f7086f8887dadacc9d9', 'request_id': 'req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.663973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] The token cache is being invalidate because project 52a6e21a32514f7086f8887dadacc9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:40.664263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.664895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-30d9ecc1-6ec7-41d9-baf7-b3dfa2c4b4de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:40.666514 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1267/2526] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:40 2022] DELETE /identity/v3/projects/52a6e21a32514f7086f8887dadacc9d9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:40.675755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.676108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.676458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.706386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:40.707546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:40.773555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1260/2527] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:40.782870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.783186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.783482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.805151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:40.806470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:40.817063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1268/2528] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:40.823678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:40.836606 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006659ad-8e06-44f4-a65b-acab25316ec6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:40.842173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:40.842890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.843189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.843484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.844670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:25:40.844998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:40.846700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:40.859801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1261/2529] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:25:40.864841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:40.875364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83566a0a-31cf-4ae4-931f-dceca941d235 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:40.880986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:40.881731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.881999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.882274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.883090 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:40.885372 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:40.914183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1269/2530] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:40 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:40.918769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:40.929329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c49790-665b-45be-be17-0d1c200a0471 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:40.987650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:40.988239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:40.988533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:40.988805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:40.989510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:40.991504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:41.013148 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1262/2531] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:40 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:25:41.017808 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:41.029159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bfbce-89ea-44cb-83d3-52605f276f35 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:41.034975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:41.035643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:41.035920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:41.036194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:41.036935 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:41.038569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:41.049906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1270/2532] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:41.054236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:41.064793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7017b1b-6427-442a-ac26-e469b852bb58 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:41.069684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:41.070348 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:41.070666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:41.070948 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:41.071683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:41.073289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d27d91-99a5-48df-81b0-e9ec208c4f92 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:41.083829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1263/2533] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:25:41.087982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:41.099262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdda84-b16c-4788-ae71-a0bd7f273b2a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:41.107296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:41.108659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:41.108975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:41.109250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] PATH_INFO: `/v3/projects/9573458ffd6449efbe0bc906ef71752e/users/dbbeac38bada4c30a56d2b27a32f290d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:41.129591 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9573458ffd6449efbe0bc906ef71752e, user_id=dbbeac38bada4c30a56d2b27a32f290d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:41.132306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:41.147723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1271/2534] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:41 2022] PUT /identity/v3/projects/9573458ffd6449efbe0bc906ef71752e/users/dbbeac38bada4c30a56d2b27a32f290d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:41.160152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:41.160439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:41.160715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:41.187697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] MFA Rules not processed for user `dbbeac38bada4c30a56d2b27a32f290d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:41.188990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:41.201986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:41.255355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1264/2535] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:41.268077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:41.270057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:41.332687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bf10c-1661-4ae1-9203-1ad985172a5a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:41.336750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:41.337395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:41.337673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:41.337948 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:41.346841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:41.348309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:41.357107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1272/2536] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:46.681927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:46.683760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:46.800450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8facdb0-ada7-4dff-8c1a-fdc948882815 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:46.808780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:46.809135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:46.809425 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:46.809775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:46.812371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:46.832484 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:46.834292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:46.846000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1265/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:51.353081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:52.005043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:52.007086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:52.073228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284fff-8e17-47e7-8c50-85e1d90ddfae service glance] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:52.077616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:52.078406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:52.078701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:52.078978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:52.081225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:52.154384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:52.155769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:52.167955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1273/2538] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:55.331195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:55.332681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:55.346305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496bf93-4903-4441-9798-2be2e5a6e822 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:55.351347 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:55.352259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:55.352547 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:55.352868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] PATH_INFO: `/v3/users/e43d573fcf384f65b7382f2d0a01ade4` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:55.354049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43d573fcf384f65b7382f2d0a01ade4)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:55.356047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:55.410047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43d573fcf384f65b7382f2d0a01ade4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.422951 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43d573fcf384f65b7382f2d0a01ade4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.432036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43d573fcf384f65b7382f2d0a01ade4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.440615 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43d573fcf384f65b7382f2d0a01ade4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.441695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-e889808d-1895-4d91-8fb6-5848b8fb4cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43d573fcf384f65b7382f2d0a01ade4'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.450358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1266/2539] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:55 2022] DELETE /identity/v3/users/e43d573fcf384f65b7382f2d0a01ade4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:25:55.637883 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:55.648172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457e440-6d2f-4b97-bce8-c63ad29b03b6 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:55.712746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:55.713565 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:55.713850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:55.714136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] PATH_INFO: `/v3/projects/77e83a80c1b34e67958aca6d0a10ba8a` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:55.715356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e83a80c1b34e67958aca6d0a10ba8a)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:55.717311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:55.771688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e83a80c1b34e67958aca6d0a10ba8a', 'request_id': 'req-201cc0eb-c99e-49d0-a205-513b0a2685d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.773553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:55.778263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e83a80c1b34e67958aca6d0a10ba8a', 'request_id': 'req-201cc0eb-c99e-49d0-a205-513b0a2685d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.787566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e83a80c1b34e67958aca6d0a10ba8a', 'request_id': 'req-201cc0eb-c99e-49d0-a205-513b0a2685d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.798293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] The token cache is being invalidate because project 77e83a80c1b34e67958aca6d0a10ba8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:55.798618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.799233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-201cc0eb-c99e-49d0-a205-513b0a2685d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:55.800818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1274/2540] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:55 2022] DELETE /identity/v3/projects/77e83a80c1b34e67958aca6d0a10ba8a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:55.806530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:55.806694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:55.807273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:55.832407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:55.834152 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:55.845616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:55.904158 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1267/2541] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:55.912940 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:55.913235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:55.913517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:55.932608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:55.933814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:55.942232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1275/2542] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:55.947089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:55.962294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb044f-2a3e-4b09-864b-21dfeaa74e18 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:55.968418 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:55.969012 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:55.969293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:55.969564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:55.970306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:25:55.970683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:55.972290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:55.985026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1268/2543] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:25:55.989622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:55.999569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9b8f5-853e-4c45-b4cf-c399c7254151 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.005657 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.006342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.006666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.006920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.007615 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.009873 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.040236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1276/2544] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:55 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:56.046909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:56.060895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f08d8-c39d-4332-8578-b4a655027480 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.119365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.120139 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.120426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.120700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.121389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.123344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.144922 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1269/2545] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:56 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:56.155169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:56.165842 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe91e3-8a35-4142-8c10-da4f1ce07916 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.170855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.171518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.171797 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.172070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.172811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.174509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.185196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1277/2546] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:56.189374 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:56.200207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad012f-e7d0-4338-bc92-de0c240ce811 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.205408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.206084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.206396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.206668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.207402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.209005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a810b-235d-48bd-b5ce-c04918f0355d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.219911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1270/2547] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:56.224175 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:56.235315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe52b6-aea8-4deb-bd1c-fbf914695878 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.241083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.241981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.242265 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.242654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] PATH_INFO: `/v3/projects/99c462317caf407094f11039ce3624e5/users/e0fab025dfe44c508d6155cf4d4aeaf4/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.260396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c462317caf407094f11039ce3624e5, user_id=e0fab025dfe44c508d6155cf4d4aeaf4, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.263263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.278420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1278/2548] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:56 2022] PUT /identity/v3/projects/99c462317caf407094f11039ce3624e5/users/e0fab025dfe44c508d6155cf4d4aeaf4/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:56.283651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e4f83-9fed-4c85-834d-8981f0f5469b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.283945 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e4f83-9fed-4c85-834d-8981f0f5469b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.284244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e4f83-9fed-4c85-834d-8981f0f5469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.302652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-490e4f83-9fed-4c85-834d-8981f0f5469b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:56.303849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-490e4f83-9fed-4c85-834d-8981f0f5469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:56.372464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1271/2549] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:56.383668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:56.386035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:56.444137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de8f35-56b5-47d6-b443-511a7d66440e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:56.448342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:56.448984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:56.449261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:56.449516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:56.459812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:56.461289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:56.471722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1279/2550] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:57.117119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.117462 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.117733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.146258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] MFA Rules not processed for user `e0fab025dfe44c508d6155cf4d4aeaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:57.147511 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:57.156265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:57.219056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1272/2551] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:57.223829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.233824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c62be5-3e1b-4bf3-a5f6-d31c50701f57 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.238833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.239429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.239715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.239989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.240685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.242621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.274328 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1280/2552] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:25:57 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:25:57.275702 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.287073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0cbe-11a8-4346-b12d-c4a3582687be None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.344262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.344956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.345234 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.345509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.346196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.348145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.369270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1273/2553] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:25:57 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:25:57.373489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.384558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118e0c-1033-4bc4-a84a-2daf3cbf1aa6 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.389896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.390606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.390895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.391174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.391915 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.393523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.405816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1281/2554] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:57.410153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.420627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bf187-9155-4ae0-9c3f-c3699d9f3765 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.426825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.427722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.428002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.428281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] PATH_INFO: `/v3/projects/eaed6c279a7f46caadb05de4db59a0e9/users/e9868843ccd34cea9c2d965df2c5a054/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.443676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaed6c279a7f46caadb05de4db59a0e9, user_id=e9868843ccd34cea9c2d965df2c5a054, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.446436 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.461838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1274/2555] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:57 2022] PUT /identity/v3/projects/eaed6c279a7f46caadb05de4db59a0e9/users/e9868843ccd34cea9c2d965df2c5a054/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:57.466312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.476932 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8310f-b88f-431d-98e0-a8d5fe01331f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.522043 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:57.538640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.539392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.539763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.540194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.541233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.543188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e190ad9-e973-4f9d-8871-c97bb21f1237 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.553595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1282/2556] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:25:57.558257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:57.569522 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7240339-2be7-4be4-8f3d-8d3c173a1bb9 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:57.575274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:57.576167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:57.576452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:57.576728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] PATH_INFO: `/v3/projects/eaed6c279a7f46caadb05de4db59a0e9/users/e9868843ccd34cea9c2d965df2c5a054/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:57.579050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaed6c279a7f46caadb05de4db59a0e9, user_id=e9868843ccd34cea9c2d965df2c5a054, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:57.581735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:57.598905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1275/2557] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:25:57 2022] PUT /identity/v3/projects/eaed6c279a7f46caadb05de4db59a0e9/users/e9868843ccd34cea9c2d965df2c5a054/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:58.246420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:58.247017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:58.247017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:58.277385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] MFA Rules not processed for user `e9868843ccd34cea9c2d965df2c5a054`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:25:58.278700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:58.351155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1283/2558] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:25:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:25:59.370546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:59.372520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:59.424505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:59.441563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:59.447107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93148a5-06dc-4803-8144-ba16cc9a27d8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:59.453370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:59.454033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:59.454307 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:59.454640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:59.456461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:59.467973 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:59.469564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:59.479779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1276/2559] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:25:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:25:59.491553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75194445-abb5-4081-9dba-a472f886d1e1 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:59.496561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:59.497532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:59.497813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:59.498121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] PATH_INFO: `/v3/users/e0fab025dfe44c508d6155cf4d4aeaf4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:59.499594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fab025dfe44c508d6155cf4d4aeaf4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:59.501531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:59.553294 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fab025dfe44c508d6155cf4d4aeaf4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.563151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fab025dfe44c508d6155cf4d4aeaf4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.563741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fab025dfe44c508d6155cf4d4aeaf4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.570670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fab025dfe44c508d6155cf4d4aeaf4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.576201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:25:59.586231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fab025dfe44c508d6155cf4d4aeaf4'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.593880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1284/2560] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:59 2022] DELETE /identity/v3/users/e0fab025dfe44c508d6155cf4d4aeaf4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:25:59.777263 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:59.790600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b6d8c-36bf-4162-8f87-0bcccc435ea3 service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:25:59.849743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:25:59.850662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:25:59.851322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:25:59.851322 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] PATH_INFO: `/v3/projects/99c462317caf407094f11039ce3624e5` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:25:59.852450 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c462317caf407094f11039ce3624e5)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:25:59.854393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:25:59.902684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c462317caf407094f11039ce3624e5', 'request_id': 'req-0b9b59b5-110e-41c7-911a-edf870ac35e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.908845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c462317caf407094f11039ce3624e5', 'request_id': 'req-0b9b59b5-110e-41c7-911a-edf870ac35e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.914396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c462317caf407094f11039ce3624e5', 'request_id': 'req-0b9b59b5-110e-41c7-911a-edf870ac35e8'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.928187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] The token cache is being invalidate because project 99c462317caf407094f11039ce3624e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:25:59.928492 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.929086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:25:59.930654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1277/2561] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:25:59 2022] DELETE /identity/v3/projects/99c462317caf407094f11039ce3624e5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:25:59.935338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:25:59.936985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:25:59.950136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b0108-44a4-46cf-aad2-879d8f8b64a7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.007365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.008241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.008526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.008804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] PATH_INFO: `/v3/users/e9868843ccd34cea9c2d965df2c5a054` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.010485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9868843ccd34cea9c2d965df2c5a054)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.012546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.070786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9868843ccd34cea9c2d965df2c5a054'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.081705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9868843ccd34cea9c2d965df2c5a054'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.082498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9868843ccd34cea9c2d965df2c5a054'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.090850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9868843ccd34cea9c2d965df2c5a054'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.103916 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-9242d8fc-9052-4f4e-a2d0-ca8ba6e5c727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9868843ccd34cea9c2d965df2c5a054'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.110259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1285/2562] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:25:59 2022] DELETE /identity/v3/users/e9868843ccd34cea9c2d965df2c5a054 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:00.305881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.307950 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:00.323026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59b5-110e-41c7-911a-edf870ac35e8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.384518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.385359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.385651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.385923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] PATH_INFO: `/v3/projects/eaed6c279a7f46caadb05de4db59a0e9` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.387124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaed6c279a7f46caadb05de4db59a0e9)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.389029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.414104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:00.440049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaed6c279a7f46caadb05de4db59a0e9', 'request_id': 'req-5d378dd1-abc9-484f-af96-32eec092024d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.446559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaed6c279a7f46caadb05de4db59a0e9', 'request_id': 'req-5d378dd1-abc9-484f-af96-32eec092024d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.453356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaed6c279a7f46caadb05de4db59a0e9', 'request_id': 'req-5d378dd1-abc9-484f-af96-32eec092024d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.468249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] The token cache is being invalidate because project eaed6c279a7f46caadb05de4db59a0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:00.468536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.469306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5d378dd1-abc9-484f-af96-32eec092024d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:00.470963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1278/2563] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:00 2022] DELETE /identity/v3/projects/eaed6c279a7f46caadb05de4db59a0e9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:00.476126 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.476667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.476766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.496495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:00.497698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:00.560443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1286/2564] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:00.569575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.569865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.570147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.589830 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:00.591058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:00.599387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1279/2565] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:00.603770 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.613668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f9f63-a353-4b93-9778-57f4b4e3b674 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.618841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.619496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.619775 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.620124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.620857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:00.621165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.622802 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.625324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:00.637893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1287/2566] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:00.642114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.651755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dd21e-9267-4583-a631-7e97bc335dfa None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.656780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.657473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.657758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.658034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.658766 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.660672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.690470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1280/2567] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:00 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:00.694767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.705494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be794-cfa0-4708-8b64-9a97528786d2 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.774490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.775184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.775473 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.775844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.777063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.780290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.807051 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1288/2568] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:00 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:00.811660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.822216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d93961-fa2d-4f10-826f-f09c27139cda admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.827275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.827944 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.828230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.828509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.829244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.830907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.841451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1281/2569] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:00.845644 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.856206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba108b2d-d88e-4c18-bcc6-e133e521a204 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.862631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.863295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.863576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.863851 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.864594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.866225 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e60c-d756-418e-979a-17855c0076aa admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.877578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1289/2570] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:00.883414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:00.894497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d411c-4706-4e68-b680-edc122c1dc33 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:00.899599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:00.900488 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.900773 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.901051 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] PATH_INFO: `/v3/projects/cccb65ef639d4c72b871f10c9350f912/users/b08d1632ef7040db84f70fd347fc61ce/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.915962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccb65ef639d4c72b871f10c9350f912, user_id=b08d1632ef7040db84f70fd347fc61ce, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:00.918758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:00.933996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1282/2571] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:00 2022] PUT /identity/v3/projects/cccb65ef639d4c72b871f10c9350f912/users/b08d1632ef7040db84f70fd347fc61ce/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:26:00.944322 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:00.944613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:00.944923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:00.974638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] MFA Rules not processed for user `b08d1632ef7040db84f70fd347fc61ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:00.975860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:01.044425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1290/2572] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:01.049323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.116258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74410-8caf-4eb4-9fd0-447dc27459bc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.122188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.122931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.123184 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.123455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.124161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.126092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.155381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1283/2573] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:01 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:01.159621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.170886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2549b1-c138-4e0f-bb5a-ed58cbc8bf00 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.229025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.229765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.230188 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.230491 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.231204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.233122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.240816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:01.256908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1291/2574] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:01 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:26:01.260623 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.271412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a5d21-99a7-43f7-be6e-f8510529e0b8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.277432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.278103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.278442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.278718 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.279448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.281054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.293205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1284/2575] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:01.297673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.308618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be812e83-d7e4-4996-920c-afd3ee801535 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.314190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.315129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.315396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.315669 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] PATH_INFO: `/v3/projects/c9571b45565d4db1a4868c0a72e7ef37/users/482e4288a51648089bfd66960b7e4029/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.330864 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9571b45565d4db1a4868c0a72e7ef37, user_id=482e4288a51648089bfd66960b7e4029, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.333513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.348499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1292/2576] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:01 2022] PUT /identity/v3/projects/c9571b45565d4db1a4868c0a72e7ef37/users/482e4288a51648089bfd66960b7e4029/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:01.352454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.362940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31bc2-e64b-4a9c-8da9-c0822900103d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.381814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:01.423163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.423818 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.424109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.424381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.425140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.426794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca3aa-0ac3-496b-b065-7076660afec1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.437308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1285/2577] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:01.441726 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.453361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455a319-9350-4513-b21e-3d8d2699ea11 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.460148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.461488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.461936 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.462354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] PATH_INFO: `/v3/projects/c9571b45565d4db1a4868c0a72e7ef37/users/482e4288a51648089bfd66960b7e4029/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.465696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9571b45565d4db1a4868c0a72e7ef37, user_id=482e4288a51648089bfd66960b7e4029, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.469868 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.483317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1293/2578] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:01 2022] PUT /identity/v3/projects/c9571b45565d4db1a4868c0a72e7ef37/users/482e4288a51648089bfd66960b7e4029/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:01.493479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.493768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.494067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.521306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] MFA Rules not processed for user `482e4288a51648089bfd66960b7e4029`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:01.522555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:01.602700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1286/2579] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:01.615216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:01.617309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:01.678428 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303426-a754-4198-820b-ecd9d521dfe1 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:01.682361 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:01.683038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:01.683318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:01.683592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:01.692839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:01.694260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:01.703946 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1294/2580] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:02.035602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.045677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aabc2f-02fe-41f8-b56a-e30e77c8d1ed None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.051302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482e4288a51648089bfd66960b7e4029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9571b45565d4db1a4868c0a72e7ef37', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.051981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.052305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.052585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.053328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:02.053640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.055294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:02.067179 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1287/2581] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:02.071453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.080793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211cb-e523-480a-980a-ff789b845c87 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.085894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482e4288a51648089bfd66960b7e4029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9571b45565d4db1a4868c0a72e7ef37', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.086572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.086861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.087139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.087846 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.089793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:02.119093 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1295/2582] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:02 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:02.123942 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.138352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acf5de-a24d-4e52-99aa-3c80a5463c87 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.200946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482e4288a51648089bfd66960b7e4029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9571b45565d4db1a4868c0a72e7ef37', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.201622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.201911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.202194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.202965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:02.203274 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.204879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:02.221217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1288/2583] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:02.260397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.271320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87889db1-d274-46d9-9b1c-7816427c7d40 tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.276364 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482e4288a51648089bfd66960b7e4029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9571b45565d4db1a4868c0a72e7ef37', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.277394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.277687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.277947 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] PATH_INFO: `/v3/projects/a334ecf29b744cc7807d2a97bfc8fd4b` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.279207 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a334ecf29b744cc7807d2a97bfc8fd4b)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.281104 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:02.332178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a334ecf29b744cc7807d2a97bfc8fd4b', 'request_id': 'req-71ab2d74-fbb8-4a90-b223-b979d697b0af'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.338877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a334ecf29b744cc7807d2a97bfc8fd4b', 'request_id': 'req-71ab2d74-fbb8-4a90-b223-b979d697b0af'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.349811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a334ecf29b744cc7807d2a97bfc8fd4b', 'request_id': 'req-71ab2d74-fbb8-4a90-b223-b979d697b0af'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.363696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] The token cache is being invalidate because project a334ecf29b744cc7807d2a97bfc8fd4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:02.363987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.364664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-71ab2d74-fbb8-4a90-b223-b979d697b0af tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.366274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1296/2584] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:02 2022] DELETE /identity/v3/projects/a334ecf29b744cc7807d2a97bfc8fd4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:02.571906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.574083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:02.603829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:02.642648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e98cc9-f4bc-4450-a32a-e81981ae17bd tempest-VolumeQuotaClassesTest-1206493212 tempest-VolumeQuotaClassesTest-1206493212-project-admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.705772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.706714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.707003 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.707279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] PATH_INFO: `/v3/users/b08d1632ef7040db84f70fd347fc61ce` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.709185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08d1632ef7040db84f70fd347fc61ce)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.712375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:02.767710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08d1632ef7040db84f70fd347fc61ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.779905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08d1632ef7040db84f70fd347fc61ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.787985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08d1632ef7040db84f70fd347fc61ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.795278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08d1632ef7040db84f70fd347fc61ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.795904 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08d1632ef7040db84f70fd347fc61ce'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:02.802068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1289/2585] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:02 2022] DELETE /identity/v3/users/b08d1632ef7040db84f70fd347fc61ce => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:02.807744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.808536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.808536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.827391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:02.828751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:02.842702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:02.897105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1297/2586] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:02.909849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:02.912112 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:02.976196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0914-055d-40bd-8d9a-ba773b664d9f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:02.980355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:02.981028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:02.981283 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:02.981557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:02.987963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:02.992932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:02.994594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:03.005997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1290/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:03.299066 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:03.300572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:03.315249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382be3cb-a342-4243-bb69-df92d7429253 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:03.321237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:03.322236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:03.322767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:03.323167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] PATH_INFO: `/v3/projects/cccb65ef639d4c72b871f10c9350f912` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:03.324986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccb65ef639d4c72b871f10c9350f912)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:03.327378 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:03.378667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccb65ef639d4c72b871f10c9350f912', 'request_id': 'req-3c519e7d-5cea-44f8-81c5-b1731ba520f0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.385354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccb65ef639d4c72b871f10c9350f912', 'request_id': 'req-3c519e7d-5cea-44f8-81c5-b1731ba520f0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.394272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccb65ef639d4c72b871f10c9350f912', 'request_id': 'req-3c519e7d-5cea-44f8-81c5-b1731ba520f0'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.405038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] The token cache is being invalidate because project cccb65ef639d4c72b871f10c9350f912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:03.405318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.405949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.407581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1298/2588] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:03 2022] DELETE /identity/v3/projects/cccb65ef639d4c72b871f10c9350f912 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:03.411947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:03.422356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6961b-7c4b-4208-8347-60bd25f087c7 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:03.481967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:03.482907 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:03.483219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:03.483497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] PATH_INFO: `/v3/users/482e4288a51648089bfd66960b7e4029` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:03.484798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] RBAC: Authorizing `identity:delete_user(user_id=482e4288a51648089bfd66960b7e4029)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:03.486791 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:03.537703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482e4288a51648089bfd66960b7e4029'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.551382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482e4288a51648089bfd66960b7e4029'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.560906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482e4288a51648089bfd66960b7e4029'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.571877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482e4288a51648089bfd66960b7e4029'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.572578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a51c9fd4-4b80-4e38-ae19-3e55ca69a46b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482e4288a51648089bfd66960b7e4029'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:03.579809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1291/2589] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:03 2022] DELETE /identity/v3/users/482e4288a51648089bfd66960b7e4029 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:03.869020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:03.870516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:03.883718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c519e7d-5cea-44f8-81c5-b1731ba520f0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:03.942564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:03.943432 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:03.943709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:03.943981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] PATH_INFO: `/v3/projects/c9571b45565d4db1a4868c0a72e7ef37` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:03.945159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9571b45565d4db1a4868c0a72e7ef37)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:03.947094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:03.990610 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:03.994566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9571b45565d4db1a4868c0a72e7ef37', 'request_id': 'req-d3710832-a493-4473-9e6b-4b5221a02094'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:04.001803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9571b45565d4db1a4868c0a72e7ef37', 'request_id': 'req-d3710832-a493-4473-9e6b-4b5221a02094'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:04.009733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9571b45565d4db1a4868c0a72e7ef37', 'request_id': 'req-d3710832-a493-4473-9e6b-4b5221a02094'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:04.020630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] The token cache is being invalidate because project c9571b45565d4db1a4868c0a72e7ef37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:04.020939 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:04.021554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-d3710832-a493-4473-9e6b-4b5221a02094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:04.023143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1299/2590] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:03 2022] DELETE /identity/v3/projects/c9571b45565d4db1a4868c0a72e7ef37 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:04.029965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.030251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.030553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.048827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:04.050058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:04.113877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1292/2591] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:04.121892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.122183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.122500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.142260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:04.143505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:04.152918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1300/2592] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:04.157397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.167908 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddd96c-5dbd-4a2b-8d3d-97799bcba9a0 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.173630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.174291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.174605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.174890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.175616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:04.175927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.177514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.186714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:04.192507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1293/2593] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:04.197254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.206705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fbfa-5043-4bb9-8caf-8e847bf5a61e None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.212603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.213266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.213550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.213828 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.214611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.216531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.245440 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1301/2594] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:04 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:04.252890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.270551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820704-e5a3-404d-8fef-71c51334da5e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.332461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.333131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.333417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.333696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.334410 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.336411 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.365195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1294/2595] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:04 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:04.369537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.380338 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf19a-6c74-4db5-849b-335204407dc0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.385970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.386905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.387034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.387312 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.388049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.389680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.400123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1302/2596] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:04.404240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.415441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2b79-5c51-4750-af67-038b0b78e13a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.420810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.421471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.421753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.422056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.422811 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.424407 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72daf-1829-43a7-a657-8cb16b416882 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.435226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1295/2597] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:04.439355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.449847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211078a-7303-43a8-9c44-deeaeac4488e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.454830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.455705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.455989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.456266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] PATH_INFO: `/v3/projects/86dfeaf2227c4c79b63901fc3d7fe1f1/users/7955fc7232c54fb98dfcce9ea2d3aa93/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.471137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dfeaf2227c4c79b63901fc3d7fe1f1, user_id=7955fc7232c54fb98dfcce9ea2d3aa93, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.473764 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.488911 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1303/2598] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:04 2022] PUT /identity/v3/projects/86dfeaf2227c4c79b63901fc3d7fe1f1/users/7955fc7232c54fb98dfcce9ea2d3aa93/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:04.494351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d55a-60dc-4ad6-a7a7-246011b1de5a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.494677 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d55a-60dc-4ad6-a7a7-246011b1de5a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.494946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d55a-60dc-4ad6-a7a7-246011b1de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.514248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-f907d55a-60dc-4ad6-a7a7-246011b1de5a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:04.515499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f907d55a-60dc-4ad6-a7a7-246011b1de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:04.580957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1296/2599] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:04.591411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:04.593231 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:04.654725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d9fb0-1586-4e82-8215-a658c883f1e5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:04.658549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:04.659199 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:04.659480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:04.659783 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:04.669110 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:04.670580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:04.679793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1304/2600] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:05.308251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.308553 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.308828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.334707 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] MFA Rules not processed for user `7955fc7232c54fb98dfcce9ea2d3aa93`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:05.335918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:05.405681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1297/2601] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:05.410278 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.420563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b4c-2f1a-4ecc-8805-f80d6eb0626a service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.425102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.425794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.426139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.426506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.427264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.429245 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.456427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1305/2602] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:05 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:05.460259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.471941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33aa1c-db8b-4e07-b8ae-d4d2f79e9b85 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.476827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:05.528888 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.529676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.529967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.530235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.530970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.532909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.554432 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1298/2603] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:05 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:26:05.558664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.569122 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157b3e7-e063-4021-aaec-145abab8c59c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.574316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.575008 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.575290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.575583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.576316 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.577901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.588528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1306/2604] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:05.593402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.608554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da16d8-a94f-4fa8-a4f5-50b1e5e802a8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.617211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.618591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.618877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.619210 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] PATH_INFO: `/v3/projects/74b14cf98e21444d9d3271c3caccb5b6/users/ce759badf69b4d4e8ddf31a721bdd5e9/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.636863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b14cf98e21444d9d3271c3caccb5b6, user_id=ce759badf69b4d4e8ddf31a721bdd5e9, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.639591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.655947 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1299/2605] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:05 2022] PUT /identity/v3/projects/74b14cf98e21444d9d3271c3caccb5b6/users/ce759badf69b4d4e8ddf31a721bdd5e9/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:05.660030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.670747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f76e0-021b-4d15-8fa3-eb369259271c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.711481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:05.736801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.737494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.737776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.738053 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.738826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.740420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d9daf-faf3-4847-94ee-65584da4ed13 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.750342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1307/2606] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:05.754724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:05.766127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6aa7-a4f1-4e07-a8ae-41f0c2926fa2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:05.771429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:05.772335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:05.772634 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:05.772913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] PATH_INFO: `/v3/projects/74b14cf98e21444d9d3271c3caccb5b6/users/ce759badf69b4d4e8ddf31a721bdd5e9/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:05.775164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b14cf98e21444d9d3271c3caccb5b6, user_id=ce759badf69b4d4e8ddf31a721bdd5e9, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:05.777921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:05.789106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1300/2607] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:05 2022] PUT /identity/v3/projects/74b14cf98e21444d9d3271c3caccb5b6/users/ce759badf69b4d4e8ddf31a721bdd5e9/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:06.441080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:06.441628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:06.441628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:06.472971 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] MFA Rules not processed for user `ce759badf69b4d4e8ddf31a721bdd5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:06.474215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:06.548721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1308/2608] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:06.560758 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:06.562653 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:06.623845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d4d38-48c7-4f41-8473-9e0aae814036 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:06.628750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:06.629715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:06.630209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:06.630703 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:06.693876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:06.695440 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:06.705514 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1301/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:08.070625 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:08.081850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da803e-6eb3-41bb-a027-f918488d9b8b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:08.085792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:08.086529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:08.086814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:08.087088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:08.098208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:08.100034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78259da4-06ae-4487-b09f-5d9db6c4c6bc service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:08.110359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1309/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:08.466069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:08.467863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:08.488088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:08.538448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76eb9-9b64-4fd5-b645-f0a4f62abb81 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:08.543805 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:08.544915 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:08.545201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:08.545477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] PATH_INFO: `/v3/users/dbbeac38bada4c30a56d2b27a32f290d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:08.547031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbeac38bada4c30a56d2b27a32f290d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:08.549124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:08.597282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbeac38bada4c30a56d2b27a32f290d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:08.611166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbeac38bada4c30a56d2b27a32f290d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:08.619161 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbeac38bada4c30a56d2b27a32f290d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:08.627195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbeac38bada4c30a56d2b27a32f290d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:08.627983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbeac38bada4c30a56d2b27a32f290d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:08.634793 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1302/2611] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:08 2022] DELETE /identity/v3/users/dbbeac38bada4c30a56d2b27a32f290d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:08.639376 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:08.639682 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:08.639959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:08.657879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:08.659169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:08.726407 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1310/2612] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:08.736617 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:08.738174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:08.808372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbca62-bdae-44be-9500-31108b0001d8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:08.813568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:08.814229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:08.814550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:08.814825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:08.824564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:08.826181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c692c7-0b61-4976-9216-27d0956f027b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:08.835810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1303/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:09.127365 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.128863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:09.142578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e73226-0909-468d-ad4d-66c6132c824d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.147688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.148910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.149229 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.149635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] PATH_INFO: `/v3/projects/9573458ffd6449efbe0bc906ef71752e` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.151635 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9573458ffd6449efbe0bc906ef71752e)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.153792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.198892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:09.207267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9573458ffd6449efbe0bc906ef71752e', 'request_id': 'req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:09.213810 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9573458ffd6449efbe0bc906ef71752e', 'request_id': 'req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:09.222640 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9573458ffd6449efbe0bc906ef71752e', 'request_id': 'req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:09.233119 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] The token cache is being invalidate because project 9573458ffd6449efbe0bc906ef71752e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:09.233405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:09.234012 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-8c154ba9-26b2-4aa6-abfa-b9c66d85aad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:09.235621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1311/2614] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:09 2022] DELETE /identity/v3/projects/9573458ffd6449efbe0bc906ef71752e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:09.240674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.240963 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.241242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.259819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:09.261045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:09.293193 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:09.322363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1304/2615] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:09.330123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.330449 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.330794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.350968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:09.352179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:09.362673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1312/2616] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:09.366936 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.377812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce639bee-9e85-4b35-bd67-59dff29da727 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.383408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.384499 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.384792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.385068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.385819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:09.386109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.387975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.401679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1305/2617] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:09.406415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.415980 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365c95-0130-4640-a940-e1f521a42286 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.421583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.422267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.422594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.422872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.423586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.425537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.458424 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1313/2618] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:09 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:09.459578 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.469975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264cc6-e770-4637-94d7-ac8b9a9e0625 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.530551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.531240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.531523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.531801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.532500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.534688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.560439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1306/2619] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:09 2022] POST /identity/v3/users => generated 280 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:26:09.561671 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.572128 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138b8f-2b68-48cb-a28d-50060695f439 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.577050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.577799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.578010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.578403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.579345 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.582562 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.594349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1314/2620] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:09.598779 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.611697 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8018a2-42ca-4c8f-997f-b6f31ff83d65 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.616802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.617604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.617845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.618235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.619134 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.620789 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccbf2-dc06-4b46-804c-6764caf327d9 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.636402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1307/2621] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:09.638639 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.655040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83525873-80c2-4887-a3fc-9630e45c7d06 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.659924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.660829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.661116 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.661397 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] PATH_INFO: `/v3/projects/35dd837b445f4660921f50139c0bc45c/users/6b71695499794b36a0009378860a688e/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.677317 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35dd837b445f4660921f50139c0bc45c, user_id=6b71695499794b36a0009378860a688e, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.680050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.694619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1315/2622] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:09 2022] PUT /identity/v3/projects/35dd837b445f4660921f50139c0bc45c/users/6b71695499794b36a0009378860a688e/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:09.763577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.763803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.763990 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.800107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] MFA Rules not processed for user `6b71695499794b36a0009378860a688e`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:09.801336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:09.872143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1308/2623] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:09.907560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:09.909724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:09.970074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dee97b-5637-4445-b47e-0f05348fa185 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:09.975016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:09.975710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:09.975995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:09.976273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:09.986505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:09.987973 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e83a1-4c45-45a6-91b8-1a24676cab70 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:09.997823 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1316/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:10.012964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:10.050489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:10.079879 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07499e-7e1e-412f-8d35-6e3c425110b6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:10.085014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:10.085903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.086188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.086491 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] PATH_INFO: `/v3/users/6b71695499794b36a0009378860a688e` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.087760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b71695499794b36a0009378860a688e)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:10.089680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:10.140010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b71695499794b36a0009378860a688e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.151889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b71695499794b36a0009378860a688e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.161198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b71695499794b36a0009378860a688e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.169142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b71695499794b36a0009378860a688e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.169839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b71695499794b36a0009378860a688e'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.176011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1309/2625] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:10 2022] DELETE /identity/v3/users/6b71695499794b36a0009378860a688e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:10.181444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.181925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.182021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.200643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:10.201861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:10.262777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1317/2626] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:10.274982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:10.277984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:10.342935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7b2aa-08e0-4856-86ba-9d51d29bb3a8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:10.347015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:10.347676 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.347957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.348265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.358428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:10.359931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67536943-f16c-483b-9b43-857b5949235a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:10.369670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1310/2627] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:26:10.657148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:10.658405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:10.671824 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1a5b-96a1-4ec2-b52a-2118e2dedb5b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:10.677557 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:10.678620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.678909 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.679187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] PATH_INFO: `/v3/projects/35dd837b445f4660921f50139c0bc45c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.680392 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35dd837b445f4660921f50139c0bc45c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:10.682506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:10.685472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:10.734169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35dd837b445f4660921f50139c0bc45c', 'request_id': 'req-84c66090-67ad-4923-91e2-05fb3843c3d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.740815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35dd837b445f4660921f50139c0bc45c', 'request_id': 'req-84c66090-67ad-4923-91e2-05fb3843c3d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.751460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35dd837b445f4660921f50139c0bc45c', 'request_id': 'req-84c66090-67ad-4923-91e2-05fb3843c3d6'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.763172 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] The token cache is being invalidate because project 35dd837b445f4660921f50139c0bc45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:10.763793 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.764143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-84c66090-67ad-4923-91e2-05fb3843c3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:10.765705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1318/2628] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:10 2022] DELETE /identity/v3/projects/35dd837b445f4660921f50139c0bc45c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:10.770983 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.771448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.771448 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.789792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:10.791018 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:10.844699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:10.858452 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1311/2629] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:10.866259 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.866651 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.866938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.887334 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:10.888619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:10.897197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1319/2630] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:10.901667 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:10.912453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69b72-6ba7-4794-8ba9-e3a331991a50 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:10.917362 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:10.918040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.918329 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.918671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.920058 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:10.920371 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:10.922016 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:10.935385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1312/2631] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:10.939819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:10.949529 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde7bc-4b1a-4b3b-847e-5171f92d5159 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:10.955306 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:10.955968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:10.956246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:10.956516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:10.957212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:10.959148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:10.989737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1320/2632] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:10 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:10.994040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:11.004394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296fe85-0dca-4a1d-a5bc-0f664d01c857 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:11.063554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:11.064227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.064510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.064785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.065482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:11.067439 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:11.089106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1313/2633] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:10 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:11.093848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:11.108187 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00593882-6c0e-4738-a7ac-e99d39ac5e84 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:11.113607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:11.114304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.114622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.114912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.115664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:11.117362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:11.129742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1321/2634] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:11.135700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:11.146998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326ea9-2f03-4b0a-87e6-240e8cb85ab0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:11.152921 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:11.153668 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.153952 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.154230 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.155028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:11.156727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c8918-2f7b-47df-a63d-04a2541d5da7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:11.167647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1314/2635] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:11.173169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:11.188220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564455-27e0-475a-b484-8bb54f6148c5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:11.194992 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:11.196202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.196498 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.196786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] PATH_INFO: `/v3/projects/d770140713d34c5ab7c8775e5e9837fa/users/3f505d44f601474aafa4ee57e71588a7/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.214340 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d770140713d34c5ab7c8775e5e9837fa, user_id=3f505d44f601474aafa4ee57e71588a7, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:11.217746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:11.233054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1322/2636] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:11 2022] PUT /identity/v3/projects/d770140713d34c5ab7c8775e5e9837fa/users/3f505d44f601474aafa4ee57e71588a7/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:26:11.238437 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac052-739a-4b11-8c28-168234b44fb1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.238849 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac052-739a-4b11-8c28-168234b44fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.239136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac052-739a-4b11-8c28-168234b44fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.258098 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-87fac052-739a-4b11-8c28-168234b44fb1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:11.259691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-87fac052-739a-4b11-8c28-168234b44fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:11.322874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1315/2637] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:11.336371 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:11.339241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:11.408369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730339-98bf-419b-9363-689689aff853 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:11.415281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:11.416189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:11.416474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:11.416792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:11.428778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:11.430257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:11.442586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1323/2638] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:12.081533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5804e4-29ed-4680-a59e-073a95d7521c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:12.082456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5804e4-29ed-4680-a59e-073a95d7521c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:12.082939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5804e4-29ed-4680-a59e-073a95d7521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:12.114122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3a5804e4-29ed-4680-a59e-073a95d7521c None None] MFA Rules not processed for user `3f505d44f601474aafa4ee57e71588a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:12.115447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3a5804e4-29ed-4680-a59e-073a95d7521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:12.171632 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:12.191984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1316/2639] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:12.214351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:12.215938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:12.272536 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:12.298131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b03778-e2d4-4c25-866e-31b5c757234c service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:12.301715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:12.302400 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:12.302677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:12.302950 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:12.313566 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:12.315045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:12.324469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1324/2640] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:14.743946 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:14.744535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:14.744535 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:14.764445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] MFA Rules not processed for user `3f505d44f601474aafa4ee57e71588a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:14.765674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:14.774269 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1317/2641] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:15.031595 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:15.034363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:15.051680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8557a8-4867-430c-b484-061e482a23df service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:15.055678 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:15.056337 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:15.056620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:15.056892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:15.058832 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:15.070148 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:15.071688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:15.080739 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1325/2642] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:15.429413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:15.431394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:15.506047 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dd0ee-c389-4da1-abb6-5795654bd559 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:15.511399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:15.512049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:15.512330 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:15.512608 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:15.514346 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:15.525665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:15.527265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:15.537062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1318/2643] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:21.404145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:21.405768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:21.421545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67965df-4e34-4daa-ab6f-c89e4837966a service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:21.426895 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:21.427778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:21.428068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:21.428349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] PATH_INFO: `/v3/users/7955fc7232c54fb98dfcce9ea2d3aa93` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:21.429579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7955fc7232c54fb98dfcce9ea2d3aa93)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:21.431528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:21.480730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7955fc7232c54fb98dfcce9ea2d3aa93'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.489247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7955fc7232c54fb98dfcce9ea2d3aa93'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.489874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7955fc7232c54fb98dfcce9ea2d3aa93'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.496890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7955fc7232c54fb98dfcce9ea2d3aa93'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.509420 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7955fc7232c54fb98dfcce9ea2d3aa93'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.515446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1326/2644] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:21 2022] DELETE /identity/v3/users/7955fc7232c54fb98dfcce9ea2d3aa93 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:21.699323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:21.710681 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825fab-d1ff-4d24-9ee5-1490c72bfc61 service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:21.769431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:21.770249 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:21.770558 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:21.770859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] PATH_INFO: `/v3/projects/86dfeaf2227c4c79b63901fc3d7fe1f1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:21.772659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dfeaf2227c4c79b63901fc3d7fe1f1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:21.774605 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:21.827581 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dfeaf2227c4c79b63901fc3d7fe1f1', 'request_id': 'req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.834151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dfeaf2227c4c79b63901fc3d7fe1f1', 'request_id': 'req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.840033 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dfeaf2227c4c79b63901fc3d7fe1f1', 'request_id': 'req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.853846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] The token cache is being invalidate because project 86dfeaf2227c4c79b63901fc3d7fe1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:21.854156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.854796 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:21.856360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1319/2645] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:21 2022] DELETE /identity/v3/projects/86dfeaf2227c4c79b63901fc3d7fe1f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:21.860701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:21.862218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:21.875584 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8068110-b702-4b23-a348-94733e6649a4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:21.933497 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:21.934402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:21.934667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:21.934963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] PATH_INFO: `/v3/users/ce759badf69b4d4e8ddf31a721bdd5e9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:21.936274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce759badf69b4d4e8ddf31a721bdd5e9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:21.938184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:21.980661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:21.992147 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce759badf69b4d4e8ddf31a721bdd5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.002121 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce759badf69b4d4e8ddf31a721bdd5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.002879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce759badf69b4d4e8ddf31a721bdd5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.010541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce759badf69b4d4e8ddf31a721bdd5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.027579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-46cbfcbd-052d-400e-b66a-f93ba23292dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce759badf69b4d4e8ddf31a721bdd5e9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.034357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1327/2646] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:21 2022] DELETE /identity/v3/users/ce759badf69b4d4e8ddf31a721bdd5e9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:22.212519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.214025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:22.226501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:22.231320 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2c4d-b1d7-4e1a-9d44-fd24c2ee9204 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.294064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.294889 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.295174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.295455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] PATH_INFO: `/v3/projects/74b14cf98e21444d9d3271c3caccb5b6` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.296638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b14cf98e21444d9d3271c3caccb5b6)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.298714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.348684 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b14cf98e21444d9d3271c3caccb5b6', 'request_id': 'req-a96fda10-9743-49a5-a78d-34a97141d373'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.354996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b14cf98e21444d9d3271c3caccb5b6', 'request_id': 'req-a96fda10-9743-49a5-a78d-34a97141d373'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.361180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b14cf98e21444d9d3271c3caccb5b6', 'request_id': 'req-a96fda10-9743-49a5-a78d-34a97141d373'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.375242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] The token cache is being invalidate because project 74b14cf98e21444d9d3271c3caccb5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:22.375580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.376182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-a96fda10-9743-49a5-a78d-34a97141d373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:22.377720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1320/2647] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:22 2022] DELETE /identity/v3/projects/74b14cf98e21444d9d3271c3caccb5b6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:22.384579 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.384869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.385146 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.404057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:22.406490 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:22.469448 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1328/2648] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:22.477661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.477962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.478239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.497600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:22.498851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:22.506836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1321/2649] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:22.511582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.521922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736e48-87df-4ac0-8da3-56b8320631cc None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.526703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.527384 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.527670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.527952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.528761 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:22.529082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.531252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.543854 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1329/2650] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:22.548598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.558591 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dd3b7-704c-4828-b1e1-9e5efd52fb51 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.563412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.564120 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.564408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.564691 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.565393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.567351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.596520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1322/2651] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:22.600803 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.611107 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b576f4-825a-47b6-8919-b43c24cc3ea3 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.676755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.677476 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.677762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.678040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.678794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.680834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.707892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1330/2652] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:22 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:22.711032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.722102 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4d51-397f-4250-9486-87b54f831d2e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.727226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.727957 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.728289 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.728568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.729296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.731130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.747426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1323/2653] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:22.748695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.760082 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75968-d764-4041-9206-f9414a2a0077 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.765604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.766590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.766897 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.767343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.768094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.769724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0dabf-8535-4b84-a594-f81bc717dc4e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.780197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1331/2654] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:22.784456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.794311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f09bd5-3247-4533-9e18-b066372b3167 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:22.798986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:22.799817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.800101 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.800377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] PATH_INFO: `/v3/projects/2a351848e9864dc5ac91d9b28a932bb9/users/835f67b604814dec983846ffccf4e3b3/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.816313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a351848e9864dc5ac91d9b28a932bb9, user_id=835f67b604814dec983846ffccf4e3b3, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:22.819042 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:22.834386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1324/2655] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:22 2022] PUT /identity/v3/projects/2a351848e9864dc5ac91d9b28a932bb9/users/835f67b604814dec983846ffccf4e3b3/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:22.844788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:22.845310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:22.845674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:22.871411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] MFA Rules not processed for user `835f67b604814dec983846ffccf4e3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:22.872590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:22.877510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:22.942799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1332/2656] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:22.953601 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:22.955884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:22.992191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:23.018688 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db07ad-89d1-4716-965e-7da5fef8ec28 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:23.022673 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:23.023319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:23.023599 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:23.023877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:23.032592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:23.034308 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:23.042933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1325/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:24.155511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:24.157003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:24.231872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f38ad-40e6-434e-b6c5-266542ca5636 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:24.238240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:24.238938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:24.239227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:24.239505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:24.241870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:24.255049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:24.256607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9e636-7c9e-4176-8475-9f208f629150 service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:24.268653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1333/2658] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:26.087256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:26.149884 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62a91-119f-402c-982b-ac0aeacf163b service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:26.154678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:26.155542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:26.155820 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:26.156123 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] PATH_INFO: `/v3/users/835f67b604814dec983846ffccf4e3b3` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:26.157355 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835f67b604814dec983846ffccf4e3b3)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:26.159358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:26.211843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835f67b604814dec983846ffccf4e3b3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.224268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835f67b604814dec983846ffccf4e3b3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.234153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835f67b604814dec983846ffccf4e3b3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.241867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835f67b604814dec983846ffccf4e3b3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.242568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835f67b604814dec983846ffccf4e3b3'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.248597 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1326/2659] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:26 2022] DELETE /identity/v3/users/835f67b604814dec983846ffccf4e3b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:26.253620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:26.254141 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:26.254388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:26.272859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:26.274725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:26.343201 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1334/2660] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:26.354828 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:26.357191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:26.436443 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4075c-1de0-49d1-9bf7-ed3e907af354 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:26.440696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:26.441428 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:26.441714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:26.442314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:26.456661 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:26.459438 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:26.461406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedb7c3-7766-4f71-a868-04945b435c0e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:26.474639 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1327/2661] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:26.776031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:26.777760 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:26.792628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611abd2d-0811-448c-b25c-95c6a3d0f822 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:26.798193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:26.800181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:26.800459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:26.800753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] PATH_INFO: `/v3/projects/2a351848e9864dc5ac91d9b28a932bb9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:26.802083 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a351848e9864dc5ac91d9b28a932bb9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:26.804752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:26.840096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:26.857120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a351848e9864dc5ac91d9b28a932bb9', 'request_id': 'req-0368f07a-e77a-46f0-9866-3524b37df381'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.863755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a351848e9864dc5ac91d9b28a932bb9', 'request_id': 'req-0368f07a-e77a-46f0-9866-3524b37df381'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.872429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a351848e9864dc5ac91d9b28a932bb9', 'request_id': 'req-0368f07a-e77a-46f0-9866-3524b37df381'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.883777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] The token cache is being invalidate because project 2a351848e9864dc5ac91d9b28a932bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:26.884351 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.885103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0368f07a-e77a-46f0-9866-3524b37df381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:26.886822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1335/2662] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:26 2022] DELETE /identity/v3/projects/2a351848e9864dc5ac91d9b28a932bb9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:26.892760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:26.893095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:26.893417 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:26.913827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:26.915294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:26.979170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1328/2663] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:27.040754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.041036 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.041304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.061702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:27.062920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:27.071695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1336/2664] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:27.076726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.088254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c4844-46a5-49b1-9c38-47510d0a5aef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.093294 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.093977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.094259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.094583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.095305 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:27.095662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.097574 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.109399 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1329/2665] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:27.113765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.123421 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8dcec-b62a-4453-a3eb-63056443bfca None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.129193 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.129561 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.129931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.130211 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.131073 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.132878 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.165599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1337/2666] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:27 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:27.166802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.177280 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba44-d0e7-4e3c-ace4-3b5426a10dc7 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.238580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.239275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.239710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.239995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.240694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.242672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.269209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1330/2667] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:27 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:27.270419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.281061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2ac87-c572-416a-ab9a-a56a53e474ee admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.286319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.287035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.287318 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.287598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.288343 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.289921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.301046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1338/2668] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:27.305297 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.315881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9a752-3463-44de-92fc-5ff9310a8c46 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.320482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.321150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.321449 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.321743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.322537 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.324094 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f772-b45f-4105-9f16-c1827167dac7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.334382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1331/2669] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:27.338643 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.349145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed23c-d60c-4930-8a12-93f44dde3969 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.355214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.356370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.356662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.356944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] PATH_INFO: `/v3/projects/15ebf4f619dc42f388da7180a9c8709f/users/0e83628438144b0fbd80b0a4da91568a/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.372989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15ebf4f619dc42f388da7180a9c8709f, user_id=0e83628438144b0fbd80b0a4da91568a, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.375901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.392096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1339/2670] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:27 2022] PUT /identity/v3/projects/15ebf4f619dc42f388da7180a9c8709f/users/0e83628438144b0fbd80b0a4da91568a/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:27.402592 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.402885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.403170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.435671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] MFA Rules not processed for user `0e83628438144b0fbd80b0a4da91568a`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:27.438055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:27.491561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:27.515652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1332/2671] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:27.527096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:27.529206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:27.607847 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831ad4-6622-4fbd-941e-b187be2ff750 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:27.611667 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:27.612315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:27.612593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:27.612867 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:27.621237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:27.622695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da454d6b-d3ef-443b-808d-370ab19747f1 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:27.632103 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1340/2672] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:30.195589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:30.270074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007e0d-5846-40ac-87b1-1adeeeb17c2c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:30.278078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:30.279457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:30.279815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:30.280163 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] PATH_INFO: `/v3/users/0e83628438144b0fbd80b0a4da91568a` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:30.282195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e83628438144b0fbd80b0a4da91568a)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:30.284810 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:30.343607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e83628438144b0fbd80b0a4da91568a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.363187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e83628438144b0fbd80b0a4da91568a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.373588 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e83628438144b0fbd80b0a4da91568a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.383424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e83628438144b0fbd80b0a4da91568a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.385091 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e83628438144b0fbd80b0a4da91568a'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.392211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1333/2673] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:30 2022] DELETE /identity/v3/users/0e83628438144b0fbd80b0a4da91568a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:30.396673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:30.397168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:30.397285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:30.415370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:30.416656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:30.475179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:30.491495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1341/2674] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:30.501207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:30.502846 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:30.562701 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf874bf-4481-4dfd-9ba4-e646436cc826 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:30.566569 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:30.567231 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:30.567517 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:30.567798 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:30.576850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:30.578299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074429c6-3335-4177-8356-40e0ade03d20 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:30.588768 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1334/2675] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:30.878742 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:30.881191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:30.896113 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7d6a7-6c30-4a59-8b91-1224f0a70d04 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:30.901670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:30.902617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:30.902899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:30.903178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] PATH_INFO: `/v3/projects/15ebf4f619dc42f388da7180a9c8709f` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:30.904691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15ebf4f619dc42f388da7180a9c8709f)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:30.906670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:30.957248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15ebf4f619dc42f388da7180a9c8709f', 'request_id': 'req-687f648a-373f-4d89-b196-4debeeb85073'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.963426 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15ebf4f619dc42f388da7180a9c8709f', 'request_id': 'req-687f648a-373f-4d89-b196-4debeeb85073'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.972634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15ebf4f619dc42f388da7180a9c8709f', 'request_id': 'req-687f648a-373f-4d89-b196-4debeeb85073'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.983926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] The token cache is being invalidate because project 15ebf4f619dc42f388da7180a9c8709f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:30.984212 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.984870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-687f648a-373f-4d89-b196-4debeeb85073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:30.986618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1342/2676] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:30 2022] DELETE /identity/v3/projects/15ebf4f619dc42f388da7180a9c8709f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:30.994052 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:30.994426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:30.994699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.016739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:31.018067 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:31.027363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:31.081015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1335/2677] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:31.089057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.089346 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.089621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.109506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:31.110759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:31.120072 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1343/2678] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:31.124442 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.136166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d197b-1920-4cc5-a4af-6d4c9dae6ca6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.141272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.141941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.142223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.142552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.143441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:31.143750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.145352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.157829 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1336/2679] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:31.162398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.171630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edc0f1-8017-4e9e-924d-9a17a3ac2339 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.176387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.177038 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.177315 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.177598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.178281 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.180208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.209886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1344/2680] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:31 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:31.214252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.224563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d149cdf-b2ea-4c67-aa2c-dea26e10746d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.281416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.282114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.282408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.282695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.283402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.285393 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.307337 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1337/2681] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:31 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:31.311938 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.323249 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782ff3-f584-4f39-8209-380c9f0b2cbf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.328388 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.329094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.329414 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.329661 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.330401 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.332273 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.344264 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1345/2682] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:31.348585 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.359105 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d2bb-ffb5-4202-8af0-447249c2eae0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.364198 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.364970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.365256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.365533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.366270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.367919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b8714-5167-4df2-9b86-37aa76087a13 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.379794 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1338/2683] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:31.384711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.397120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a06a-86b0-489e-a1ac-5be5a37f739a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.402532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.403457 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.403747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.404034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] PATH_INFO: `/v3/projects/3e8673afc03640998d4a67840213c622/users/4cc0473a797240b6b38e7aa5528cf558/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.421564 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e8673afc03640998d4a67840213c622, user_id=4cc0473a797240b6b38e7aa5528cf558, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.424284 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.442712 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1346/2684] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:31 2022] PUT /identity/v3/projects/3e8673afc03640998d4a67840213c622/users/4cc0473a797240b6b38e7aa5528cf558/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:31.454389 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.454842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.455129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.490088 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] MFA Rules not processed for user `4cc0473a797240b6b38e7aa5528cf558`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:31.491665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:31.583590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1339/2685] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:31.590874 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:31.595582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:31.668788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70b0fa-8d16-4994-b49b-9c31ca887192 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:31.672677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:31.673664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:31.674050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:31.674493 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:31.687068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:31.688676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894ba7-83b7-4a8a-8f6b-7bc5fc129bcf service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:31.699228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1347/2686] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:32.507583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:32.509567 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:32.521649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:32.608287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbcda8-b5a6-407f-8c9c-9272ddb80ecd None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:32.616339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:32.617060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:32.617341 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:32.617616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:32.619825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:32.636025 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:32.637644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27244c43-5864-49a8-9994-9533c22830cd service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:32.650187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1340/2687] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:32.951025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:32.951481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:32.951708 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:32.969184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] MFA Rules not processed for user `3f505d44f601474aafa4ee57e71588a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:32.970480 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:32.984010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:33.031295 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1348/2688] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:36.787643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e4df-fb28-460c-bade-2fd929a54641 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:36.788153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e4df-fb28-460c-bade-2fd929a54641 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:36.788241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e4df-fb28-460c-bade-2fd929a54641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:36.811293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ad39e4df-fb28-460c-bade-2fd929a54641 None None] MFA Rules not processed for user `ccf66c5198f24f7cbc39132635c03611`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:36.812640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ad39e4df-fb28-460c-bade-2fd929a54641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:36.883457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1341/2689] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:41.978246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:42.053694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36581c-8957-46a0-b399-57bd68989feb None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:42.059597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:42.061129 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:42.061756 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:42.062254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] PATH_INFO: `/v3/users/4cc0473a797240b6b38e7aa5528cf558` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:42.063730 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc0473a797240b6b38e7aa5528cf558)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:42.066057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:42.155276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc0473a797240b6b38e7aa5528cf558'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:42.167517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc0473a797240b6b38e7aa5528cf558'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:42.168896 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc0473a797240b6b38e7aa5528cf558'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:42.181899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc0473a797240b6b38e7aa5528cf558'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:42.202157 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc0473a797240b6b38e7aa5528cf558'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:42.210474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1349/2690] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:26:41 2022] DELETE /identity/v3/users/4cc0473a797240b6b38e7aa5528cf558 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:42.215565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:42.216603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:42.217074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:42.252422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:42.253923 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:42.385350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1342/2691] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:26:42.419258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:42.421291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:42.480441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:42.495921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f8a2-e85e-4473-9c0a-393e4b19be48 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:42.501078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:42.501986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:42.502406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:42.503138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:42.518303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:42.520988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e27d3-91d0-4571-8a91-9f557983b6a7 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:42.537094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1350/2692] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:43.075545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.077555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:43.091494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf728b1-25c9-4b46-8a7b-e92819c6eef6 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.105071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.108181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.108657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.109415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] PATH_INFO: `/v3/projects/3e8673afc03640998d4a67840213c622` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.111174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e8673afc03640998d4a67840213c622)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.120028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.198659 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:43.216551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e8673afc03640998d4a67840213c622', 'request_id': 'req-6f04702e-65da-455d-b171-d9668fc61c5d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:43.225369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e8673afc03640998d4a67840213c622', 'request_id': 'req-6f04702e-65da-455d-b171-d9668fc61c5d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:43.231986 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e8673afc03640998d4a67840213c622', 'request_id': 'req-6f04702e-65da-455d-b171-d9668fc61c5d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:43.266774 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] The token cache is being invalidate because project 3e8673afc03640998d4a67840213c622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:26:43.271978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:43.273765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6f04702e-65da-455d-b171-d9668fc61c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:26:43.279275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1343/2693] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:26:43 2022] DELETE /identity/v3/projects/3e8673afc03640998d4a67840213c622 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:43.281047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.281509 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.281604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.332112 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:43.333754 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:43.470137 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1351/2694] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:43.480712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.481002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.481494 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.510746 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:43.512459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:43.530336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1344/2695] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:43.535031 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.557830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4f8c7-17e0-4e46-8a47-cd9abc7d2dd4 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.567988 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.569307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.570827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.571435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.574684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:26:43.575096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.579014 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.605569 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1352/2696] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:26:43.610205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.620336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918c65f-c5f3-484b-b10e-d14294a2c914 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.639918 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.640721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.641100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.641593 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.643172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.645687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.685850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1345/2697] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:43 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:43.691738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.704945 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6936e-b5f8-4a27-bec1-4a97e5b325c7 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.790572 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.791300 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.791587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.791894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.792620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.794668 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.823246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1353/2698] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:43 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:43.824836 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.836197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a47b6-8b41-47ee-a321-a2409aea329e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.841103 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.841760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.842045 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.842324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.843176 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.844857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.859536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1346/2699] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:26:43.860776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.883266 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402851a5-8333-4ee0-b8eb-1bae3c4bd832 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.888567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.889256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.889541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.889819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.890614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.892227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d0e7e-9870-4175-8e53-5e613740f104 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.904078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1354/2700] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:43.908156 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:43.919277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3614-f28e-4123-898e-d10b8cbfde91 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:43.924044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:43.925128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.925420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.925699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] PATH_INFO: `/v3/projects/4bb5a1e0698348739f44376f82c933ad/users/b8158ebeedf7438b9e3306186c1898cd/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.942391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb5a1e0698348739f44376f82c933ad, user_id=b8158ebeedf7438b9e3306186c1898cd, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:43.945254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:43.960687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1347/2701] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:43 2022] PUT /identity/v3/projects/4bb5a1e0698348739f44376f82c933ad/users/b8158ebeedf7438b9e3306186c1898cd/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:26:43.965716 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c6a27-4781-4bbd-b601-46776271e8f1 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:43.966140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c6a27-4781-4bbd-b601-46776271e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:43.966544 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c6a27-4781-4bbd-b601-46776271e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:43.986169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-bf1c6a27-4781-4bbd-b601-46776271e8f1 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:43.987736 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bf1c6a27-4781-4bbd-b601-46776271e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:44.041776 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:44.062174 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1355/2702] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:44.078706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:44.080497 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:44.146546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c422-ae16-4089-8791-52cd31857373 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:44.150326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:44.151030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:44.151319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:44.151611 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:44.164590 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:44.166241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:44.177696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1348/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:50.319683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.320308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.320308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.367496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] MFA Rules not processed for user `b8158ebeedf7438b9e3306186c1898cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:50.368774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:50.439560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1356/2704] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:50.445110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.455434 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d701-7145-4878-befc-8479c6663d8e service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.462180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.462890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.463170 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.463441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.464136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.466070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.499177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1349/2705] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:26:50 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:50.503683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.516892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feadf81-d267-4fd8-b9e1-27161db6c5ad None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.588323 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.589350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.589857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.590352 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.591183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.593241 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.622969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1357/2706] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:26:50 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:26:50.629524 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.650570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb59d9-5e61-421f-9686-402c373088cc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.656781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.657520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.657799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.658074 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.658804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.661132 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.678932 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1350/2707] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:50.683412 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.698587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ad54e-8cfb-4f85-a103-d3707bb776dc admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.705077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.706204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.706506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.706800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] PATH_INFO: `/v3/projects/83178ea0925a4d0caae854926a2bec25/users/58714b34e1bd4b76ae80b1a2293677da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.724047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83178ea0925a4d0caae854926a2bec25, user_id=58714b34e1bd4b76ae80b1a2293677da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.727880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.745664 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1358/2708] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:50 2022] PUT /identity/v3/projects/83178ea0925a4d0caae854926a2bec25/users/58714b34e1bd4b76ae80b1a2293677da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:50.749838 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.761563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbe39-be6a-47f1-8411-f6785b46d309 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.787445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:50.821869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.822589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.822872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.823146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.823874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.825516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.835968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1351/2709] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:26:50.840357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:50.850902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c613b7-d5be-46f6-ab82-58156e3ba2c4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:50.855533 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:50.858046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:50.858046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:50.858046 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] PATH_INFO: `/v3/projects/83178ea0925a4d0caae854926a2bec25/users/58714b34e1bd4b76ae80b1a2293677da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:50.859840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=83178ea0925a4d0caae854926a2bec25, user_id=58714b34e1bd4b76ae80b1a2293677da, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:50.863311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0a3a-0151-4a8e-8dfb-3d9d1397afeb admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:50.875070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1359/2710] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:26:50 2022] PUT /identity/v3/projects/83178ea0925a4d0caae854926a2bec25/users/58714b34e1bd4b76ae80b1a2293677da/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:26:51.038630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:51.040303 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:51.162913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dd9f4-f352-4d82-913f-2661b620c90e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:51.276582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:51.277306 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:51.278239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:51.278554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:51.281259 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:51.368211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:51.370083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:51.378022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:51.396291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1352/2711] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:57.723622 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fb67-4ccc-4aa2-8f4a-5d08755c36a6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:57.724713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fb67-4ccc-4aa2-8f4a-5d08755c36a6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:57.725202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3fb67-4ccc-4aa2-8f4a-5d08755c36a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:57.813952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fbc3fb67-4ccc-4aa2-8f4a-5d08755c36a6 None None] MFA Rules not processed for user `58714b34e1bd4b76ae80b1a2293677da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:57.815528 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fbc3fb67-4ccc-4aa2-8f4a-5d08755c36a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:57.879861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:26:57.954537 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1360/2712] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:26:57.969866 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:26:57.972429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:57.989386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21576-b87c-48f6-8265-99e265dcb61b service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:26:57.995510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:26:57.996181 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:57.996471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:57.996754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:58.062799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:26:58.064531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39251a3-94c9-47c1-bd72-a387850c53bc service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:26:58.075485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1353/2713] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:26:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:26:59.145908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:26:59.146949 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:26:59.147298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:26:59.168062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] MFA Rules not processed for user `b8158ebeedf7438b9e3306186c1898cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:26:59.169311 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:26:59.177705 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1361/2714] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:26:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:02.989895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:02.990400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:02.990400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:03.012526 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:27:03.013743 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:03.078219 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1354/2715] 10.222.0.16 () {58 vars in 1009 bytes} [Mon Jul 4 04:27:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:03.135693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:03.137102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:03.215320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a34b9-9b24-4e8f-a066-590a27c689d2 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:03.220676 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:03.221550 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:03.224866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:03.224866 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:03.231387 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:03.233034 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:03.241280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1362/2716] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:27:05.465345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:05.532860 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd379-d57e-4273-b804-7a4d925fb252 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:05.536822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:05.537519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:05.537824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:05.538119 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:05.540244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:05.616293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:05.621126 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:05.623516 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:05.639692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1355/2717] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:27:13.240711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=54 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:13.491452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:13.493430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:13.641326 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635cd1-9317-4fff-b76a-31e4fda931e3 service swift] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:13.647190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:13.648496 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:13.648788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:13.649143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] PATH_INFO: `/v3/users/34ec54486c9d404896f832b87f99bc11` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:13.651078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ec54486c9d404896f832b87f99bc11)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:13.654040 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:13.716877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ec54486c9d404896f832b87f99bc11'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:13.726194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ec54486c9d404896f832b87f99bc11'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:13.727108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ec54486c9d404896f832b87f99bc11'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:13.735075 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ec54486c9d404896f832b87f99bc11'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:13.750587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ec54486c9d404896f832b87f99bc11'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:13.758813 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1363/2718] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:27:13 2022] DELETE /identity/v3/users/34ec54486c9d404896f832b87f99bc11 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:13.976759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:14.068894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278261d9-1749-4d30-999f-43cc1ea0a01a service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:14.151664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:14.152876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:14.153335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:14.153609 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] PATH_INFO: `/v3/projects/b622ab692a73432784c63b9ad3290e72` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:14.154853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b622ab692a73432784c63b9ad3290e72)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:14.156800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:14.209471 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b622ab692a73432784c63b9ad3290e72', 'request_id': 'req-daa967e4-2a49-495a-b536-d0b56df8f7a5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.217056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b622ab692a73432784c63b9ad3290e72', 'request_id': 'req-daa967e4-2a49-495a-b536-d0b56df8f7a5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.223853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b622ab692a73432784c63b9ad3290e72', 'request_id': 'req-daa967e4-2a49-495a-b536-d0b56df8f7a5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.238709 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] The token cache is being invalidate because project b622ab692a73432784c63b9ad3290e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:27:14.239000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.239637 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.241360 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1356/2719] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:27:13 2022] DELETE /identity/v3/projects/b622ab692a73432784c63b9ad3290e72 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:14.245325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:14.246759 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:14.261003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c85e-3e82-4250-a613-5a9dac7b5cce admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:14.329598 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:14.330494 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:14.330778 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:14.331385 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] PATH_INFO: `/v3/users/e6357d98d18f435c950fdc3fe2982c5d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:14.332583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6357d98d18f435c950fdc3fe2982c5d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:14.334379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:14.348952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:14.393184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6357d98d18f435c950fdc3fe2982c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.403289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6357d98d18f435c950fdc3fe2982c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.403841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6357d98d18f435c950fdc3fe2982c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.411902 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6357d98d18f435c950fdc3fe2982c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.427171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-5fff6451-bde1-463f-bede-54456e2d53ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6357d98d18f435c950fdc3fe2982c5d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.437423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1364/2720] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:27:14 2022] DELETE /identity/v3/users/e6357d98d18f435c950fdc3fe2982c5d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:14.661816 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:14.663630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:14.698613 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa967e4-2a49-495a-b536-d0b56df8f7a5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:14.749727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:14.769474 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:14.770536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:14.770890 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:14.771235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] PATH_INFO: `/v3/projects/2585949985ce4eb199960e63235257cf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:14.772646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2585949985ce4eb199960e63235257cf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:14.774717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:14.840019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2585949985ce4eb199960e63235257cf', 'request_id': 'req-f96ff6bf-a93e-42f2-a45f-b925c7192f43'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.848647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2585949985ce4eb199960e63235257cf', 'request_id': 'req-f96ff6bf-a93e-42f2-a45f-b925c7192f43'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.863675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2585949985ce4eb199960e63235257cf', 'request_id': 'req-f96ff6bf-a93e-42f2-a45f-b925c7192f43'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.880379 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] The token cache is being invalidate because project 2585949985ce4eb199960e63235257cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:27:14.880664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.881316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:14.886412 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1357/2721] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:27:14 2022] DELETE /identity/v3/projects/2585949985ce4eb199960e63235257cf => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:24.430522 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:24.536180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:24.536467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:24.536745 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:24.566777 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] MFA Rules not processed for user `6a190fb9da064c5e9d4eea2abb9cdfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:27:24.568076 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:24.642260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1365/2722] 10.222.0.16 () {58 vars in 967 bytes} [Mon Jul 4 04:27:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:24.655350 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:24.657403 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:24.729989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ff6bf-a93e-42f2-a45f-b925c7192f43 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:24.735565 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01612217d9fb486daccdc549d9e0c6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:24.736223 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:24.736502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:24.736778 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:24.745128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:24.746741 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:24.754949 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1358/2723] 10.222.0.16 () {60 vars in 1384 bytes} [Mon Jul 4 04:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 04:27:26.752208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:26.753767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:26.820518 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2340-58e8-487d-b7ca-35cc645ac157 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:26.825236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:26.826136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:26.826418 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:26.826721 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] PATH_INFO: `/v3/users/3f505d44f601474aafa4ee57e71588a7` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:26.827908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f505d44f601474aafa4ee57e71588a7)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:26.829838 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:26.879437 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f505d44f601474aafa4ee57e71588a7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:26.888905 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f505d44f601474aafa4ee57e71588a7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:26.889593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f505d44f601474aafa4ee57e71588a7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:26.896772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f505d44f601474aafa4ee57e71588a7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:26.908662 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-68d1b79e-bc7d-4dfd-b0bb-af74b40a8cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f505d44f601474aafa4ee57e71588a7'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:26.910820 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:26.916080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1366/2724] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:27:26 2022] DELETE /identity/v3/users/3f505d44f601474aafa4ee57e71588a7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:27.110614 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.123761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15580c6-0a8d-4a6b-b6b5-054f0d48c2a7 service swift] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.185001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.185939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.186221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.186531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] PATH_INFO: `/v3/projects/d770140713d34c5ab7c8775e5e9837fa` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.187731 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d770140713d34c5ab7c8775e5e9837fa)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.189662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.196013 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:27.240554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd770140713d34c5ab7c8775e5e9837fa', 'request_id': 'req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:27.246564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd770140713d34c5ab7c8775e5e9837fa', 'request_id': 'req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:27.252813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd770140713d34c5ab7c8775e5e9837fa', 'request_id': 'req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:27.269633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] The token cache is being invalidate because project d770140713d34c5ab7c8775e5e9837fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:27:27.269928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:27.270845 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-285bd0d0-e683-4a4d-bea5-bd13e4957dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:27:27.272695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1359/2725] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:27:27 2022] DELETE /identity/v3/projects/d770140713d34c5ab7c8775e5e9837fa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:27.277757 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.278123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.278506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.335180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:27:27.336379 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:27.400830 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1367/2726] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:27.409205 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.409498 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.409775 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.430345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:27:27.431562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:27.440272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1360/2727] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:27.444593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.454500 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a369e-202a-42b4-bf3b-258c3484b3f5 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.459741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.460394 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.460680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.460956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.461696 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:27:27.462011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.463626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.475382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1368/2728] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:27:27.481351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.491029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d9e6-d70e-4c84-949c-3f6f581f6b61 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.496587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.497242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.497545 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.497817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.498555 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.500451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.535376 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1361/2729] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:27:27 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:27:27.541489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.555374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62119-7893-42ea-9510-dde3c883a64f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.613599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.614255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.614580 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.614845 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.615596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.617507 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.638484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1369/2730] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:27:27 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:27:27.642857 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.653519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1eec3-4c77-4889-9379-99cf11a75e75 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.658981 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.659646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.659954 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.660226 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.660965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.662729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.673819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1362/2731] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:27:27.677890 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.688906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346093b6-ca67-42d6-8861-310a4d183075 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.694391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.694799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.695081 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.695358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.696088 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.697672 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3654f9d-1ee2-4344-86ba-c1ad0cfae221 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.708285 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1370/2732] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:27:27.713212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.724160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27c1c9-7882-4c0d-b999-3c24f1b26700 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.730727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.731687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.731978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.732334 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] PATH_INFO: `/v3/projects/efe6c1718eb04d549138300f60d19b95/users/59a0bb7908f04282b74f6c21093c9858/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.749217 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe6c1718eb04d549138300f60d19b95, user_id=59a0bb7908f04282b74f6c21093c9858, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.751938 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.773049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1363/2733] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:27:27 2022] PUT /identity/v3/projects/efe6c1718eb04d549138300f60d19b95/users/59a0bb7908f04282b74f6c21093c9858/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:27:27.784844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.785653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.785653 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.814858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] MFA Rules not processed for user `59a0bb7908f04282b74f6c21093c9858`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:27:27.816059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:27.844250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:27.890362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1371/2734] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:27:27.902748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:27.904803 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:27.965145 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fea32e-e295-4ab5-b08a-f620349357fb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:27.969104 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:27.969759 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:27.970043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:27.970324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:27.980377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:27.982647 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:27.996970 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1364/2735] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:27:29.759677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:29.761461 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:29.850618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0386dc-addf-41eb-8744-e41b411c5b85 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:29.855593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:29.856247 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:29.856527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:29.856805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:29.858955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:29.869402 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:29.871186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:29.880700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1372/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:27:37.989874 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:39.876403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=51 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:27:48.516859 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:48.518504 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:48.531714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf27f1-7d63-4938-bfb7-662601cde665 service glance] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:48.536654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:48.537357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:48.537645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:48.537919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:48.539821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:48.607285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:48.608784 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:48.619160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1365/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:27:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:27:50.773888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:27:50.775665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:50.903475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da02d-b24c-443c-8ebe-212616d2bc0d service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:27:50.909573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:27:50.910422 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:27:50.910814 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:27:50.911599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:27:50.913910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:27:50.932433 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:27:50.935709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5b0ea-9829-4115-9298-8eb4650f5a85 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:27:50.951086 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1373/2738] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:27:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:27:58.614531 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=30 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:00.945628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=41 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:13.269633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:13.336568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d84d-4ace-455f-8933-e414ff01eaaa service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:13.342130 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:13.343056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:13.343363 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:13.343628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] PATH_INFO: `/v3/users/59a0bb7908f04282b74f6c21093c9858` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:13.345035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a0bb7908f04282b74f6c21093c9858)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:13.346994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:13.406670 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a0bb7908f04282b74f6c21093c9858'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:13.421782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a0bb7908f04282b74f6c21093c9858'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:13.435150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a0bb7908f04282b74f6c21093c9858'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:13.443153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a0bb7908f04282b74f6c21093c9858'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:13.443780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a0bb7908f04282b74f6c21093c9858'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:13.450252 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1366/2739] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:28:13 2022] DELETE /identity/v3/users/59a0bb7908f04282b74f6c21093c9858 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:28:13.455087 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:13.455596 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:13.455701 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:13.474425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:13.475725 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:13.543362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1374/2740] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:13.553265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:13.554916 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:13.621583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b5846-be54-484c-84f3-dcd1d77156cc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:13.626089 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088021-f980-49de-a09b-91b86181035b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:13.626870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088021-f980-49de-a09b-91b86181035b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:13.627151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088021-f980-49de-a09b-91b86181035b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:13.627420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088021-f980-49de-a09b-91b86181035b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:13.638464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088021-f980-49de-a09b-91b86181035b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:13.639911 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088021-f980-49de-a09b-91b86181035b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:13.649727 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1367/2741] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:28:13.916604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:13.918237 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:13.933386 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3c351-5434-41df-a2f8-c11bdd77cd4f None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:13.950880 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:13.951659 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:13.951975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:13.952210 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] PATH_INFO: `/v3/projects/efe6c1718eb04d549138300f60d19b95` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:13.953510 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe6c1718eb04d549138300f60d19b95)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:13.955474 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.005327 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe6c1718eb04d549138300f60d19b95', 'request_id': 'req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:14.011963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe6c1718eb04d549138300f60d19b95', 'request_id': 'req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:14.020779 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe6c1718eb04d549138300f60d19b95', 'request_id': 'req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:14.031665 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] The token cache is being invalidate because project efe6c1718eb04d549138300f60d19b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:28:14.031954 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:14.032617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cd9bbe60-1d85-4210-abb3-f81cce0e05a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:28:14.034405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1375/2742] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:28:13 2022] DELETE /identity/v3/projects/efe6c1718eb04d549138300f60d19b95 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:28:14.039495 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.039780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.040056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.058256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:14.059433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:14.076881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:14.123186 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1368/2743] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:14.131666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.131957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.132269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.152807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:14.154017 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:14.163324 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1376/2744] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:14.169293 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.179664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489340ab-b0bc-4060-b4a2-68b066db054e None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.185128 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.185792 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.186071 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.186340 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.187106 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:28:14.187408 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.189010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.201257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1369/2745] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:28:14.205750 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.215234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2fe5d-d794-4010-9fcf-7efdb1e379e7 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.220786 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.221450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.221731 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.222004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.222737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.224649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.257370 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1377/2746] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:28:14 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:28:14.262580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.276509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deac21-6151-44e2-86b0-c87589515224 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.343370 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.343984 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.344232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.344518 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.345202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.347150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.373967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1370/2747] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:28:14 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:28:14.378836 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.396252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea8fad-db32-40e4-9c0e-7c1b6dbf0519 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.402028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.402737 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.403003 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.403282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.404024 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.405620 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.416747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1378/2748] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:28:14.422165 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.432894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c04cc8-98c0-455c-a5d3-997dfd2a2dc1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.438559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.439241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.439530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.439812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.440548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.442141 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f7bec-35e9-436c-b04d-03c2385d55ac admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.455151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1371/2749] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:28:14.459870 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.471309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64299da-dc2c-4cf3-bbbe-b9347d620d20 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.476619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.477506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.477788 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.478065 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] PATH_INFO: `/v3/projects/5cfc2721cf3146beb4471ded02b1b637/users/4e12c4cfe11c4be58991df7173e4533f/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.492812 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cfc2721cf3146beb4471ded02b1b637, user_id=4e12c4cfe11c4be58991df7173e4533f, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.495555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.511192 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1379/2750] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:28:14 2022] PUT /identity/v3/projects/5cfc2721cf3146beb4471ded02b1b637/users/4e12c4cfe11c4be58991df7173e4533f/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:28:14.516354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e3cc-2a10-4970-afe7-da31e5362123 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.516648 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e3cc-2a10-4970-afe7-da31e5362123 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.516930 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e3cc-2a10-4970-afe7-da31e5362123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.535570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-12a2e3cc-2a10-4970-afe7-da31e5362123 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:14.536804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12a2e3cc-2a10-4970-afe7-da31e5362123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:14.603754 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1372/2751] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:14.613468 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:14.615405 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:14.639979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:14.680597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ab46-e0cb-4f3a-a1d5-353b6834f61a admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:14.685084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:14.685746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:14.686030 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:14.686342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:14.696839 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:14.698263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:14.708069 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1380/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:28:15.619073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430a51-ac3e-4bbe-829b-f3f80f10d749 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:15.619640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430a51-ac3e-4bbe-829b-f3f80f10d749 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:15.619640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42430a51-ac3e-4bbe-829b-f3f80f10d749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:15.650630 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-42430a51-ac3e-4bbe-829b-f3f80f10d749 None None] MFA Rules not processed for user `4e12c4cfe11c4be58991df7173e4533f`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:15.651864 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-42430a51-ac3e-4bbe-829b-f3f80f10d749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:15.668902 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:15.718129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1373/2753] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:15.729356 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:28:15.730790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:15.786825 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067d4d9-2a9c-4c60-b06c-1a4839d1fc96 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:28:15.790904 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:28:15.791571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:15.791857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:15.792131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:15.800711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:28:15.802155 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:28:15.811446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1381/2754] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:28:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:28:17.182823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:28:17.183344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:28:17.183457 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:28:17.201607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] MFA Rules not processed for user `4e12c4cfe11c4be58991df7173e4533f`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:28:17.202877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:28:17.210817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1374/2755] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:28:25.807466 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=43 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:28:27.196100 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=22 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:31.382208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:31.384453 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:31.398799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9405-0244-42ba-aa93-207ac6be7858 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:31.403310 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:31.403964 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:31.404246 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:31.404521 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:31.406444 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:31.420353 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:31.421821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:31.432304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1382/2756] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:34.434200 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:34.436291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:34.465290 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6dadb-ef62-4979-88ce-6c11f27de4ef None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:34.470482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:34.471444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:34.471733 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:34.472010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] PATH_INFO: `/v3/users/ccf66c5198f24f7cbc39132635c03611` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:34.473218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf66c5198f24f7cbc39132635c03611)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:34.475248 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:34.527324 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf66c5198f24f7cbc39132635c03611'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.540404 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf66c5198f24f7cbc39132635c03611'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.550559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf66c5198f24f7cbc39132635c03611'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.558998 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf66c5198f24f7cbc39132635c03611'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.559674 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-fd497777-ea97-45f0-9095-232efe232f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf66c5198f24f7cbc39132635c03611'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.566124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1375/2757] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:34 2022] DELETE /identity/v3/users/ccf66c5198f24f7cbc39132635c03611 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:34.746899 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:34.758677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954f1d3-fd78-407a-8206-1a1cfa40c8a1 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:34.820628 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:34.821492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:34.821772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:34.822047 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] PATH_INFO: `/v3/projects/0bd3f8996b5b46f6904e32023b092e5d` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:34.823267 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd3f8996b5b46f6904e32023b092e5d)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:34.825160 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:34.879026 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd3f8996b5b46f6904e32023b092e5d', 'request_id': 'req-344db2bb-6563-438b-ba93-69f36a75aa9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.886001 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd3f8996b5b46f6904e32023b092e5d', 'request_id': 'req-344db2bb-6563-438b-ba93-69f36a75aa9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.896924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd3f8996b5b46f6904e32023b092e5d', 'request_id': 'req-344db2bb-6563-438b-ba93-69f36a75aa9d'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.907753 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] The token cache is being invalidate because project 0bd3f8996b5b46f6904e32023b092e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:34.908045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.908755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-344db2bb-6563-438b-ba93-69f36a75aa9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:34.910467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1383/2758] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:34 2022] DELETE /identity/v3/projects/0bd3f8996b5b46f6904e32023b092e5d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:34.915901 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:34.916213 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:34.916489 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:34.935881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:34.937085 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:34.998704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1376/2759] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:35.006649 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.006941 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.007218 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.026607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:35.027818 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:35.036341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1384/2760] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:35.040696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.051387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de559dc-29b9-48c2-bc87-d5fab9759b5a None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.057024 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.057700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.057985 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.058257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.060174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:29:35.060174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.061019 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.074178 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1377/2761] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:29:35.078656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.088446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c29d-c976-4a2f-80b2-a84e7684b18c None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.093502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.094163 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.094459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.094747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.095446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.097336 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.125608 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1385/2762] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:35.129994 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.141209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e2204-fcb2-4300-8925-22c8ca61ac62 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.199108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.199802 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.200092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.200369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.201086 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.203063 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.209685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:35.230541 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1378/2763] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:29:35.231680 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.242234 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fd0ce-24c0-4184-9664-b37be2fdfbee admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.247389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.248057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.248341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.248614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.249515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.251139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.262837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1386/2764] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:35.267188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.277397 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919a50-ca9a-4c87-af77-4a3308b57f65 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.283813 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.284800 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.285059 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.285490 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.286640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.289084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ffbc2-d881-429c-9750-a67f4c5fbf1d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.302118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1379/2765] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 04:29:35.306637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.317106 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67f27-9cfd-4d8f-9b47-b7c52d45f59d admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.323004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.323892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.324169 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.324443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] PATH_INFO: `/v3/projects/41f84fc7c5644b438a9271a9c4b66edf/users/acd5a0db2752455e82c17d4a8cfe111b/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.340688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f84fc7c5644b438a9271a9c4b66edf, user_id=acd5a0db2752455e82c17d4a8cfe111b, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.343357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.361548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1387/2766] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:35 2022] PUT /identity/v3/projects/41f84fc7c5644b438a9271a9c4b66edf/users/acd5a0db2752455e82c17d4a8cfe111b/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:35.373975 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.374265 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.374587 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.402519 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] MFA Rules not processed for user `acd5a0db2752455e82c17d4a8cfe111b`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:35.403730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:35.470447 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1380/2767] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:35.475587 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.490617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:35.542423 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5494794-e314-4027-a5dd-19aa908a3aa5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.547594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.548262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.548549 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.548826 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.549516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.551467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.588419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1388/2768] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:35.592577 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.604472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9db72-982d-4195-89e4-e7ea8ce2a69b None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.681795 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.682551 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.682837 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.683114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.683955 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.685878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.708150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1381/2769] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:29:35.712445 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.723434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fef599-06aa-41ad-8037-93ec0861bef8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.728922 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.729629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.729908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.730233 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.731068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.732673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.744914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1389/2770] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:35.749523 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.761050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127daa53-306b-40fc-8194-dbe0c2288f6a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.768010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.768909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.769190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.769458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] PATH_INFO: `/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9/users/5e8f7c868edf4f939b59bef3f0afb2ce/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.785201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac5edbd6d824a8a9ba08303f4f34ad9, user_id=5e8f7c868edf4f939b59bef3f0afb2ce, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.788073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.803571 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1382/2771] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:35 2022] PUT /identity/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9/users/5e8f7c868edf4f939b59bef3f0afb2ce/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:35.808515 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.819220 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf66a3-f4cf-4e41-8025-9664c777d829 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.877011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.877693 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.877975 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.878277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.878995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.880681 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673ceaf-0ead-4641-ad11-c6fdcaf988ee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.891269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1390/2772] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:35.896202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:35.910996 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbef1a5-0cdf-463d-a19f-e4d69da3fb75 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:35.917349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:35.918236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.918548 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.918825 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] PATH_INFO: `/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9/users/5e8f7c868edf4f939b59bef3f0afb2ce/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.920881 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac5edbd6d824a8a9ba08303f4f34ad9, user_id=5e8f7c868edf4f939b59bef3f0afb2ce, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:35.924685 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:35.935270 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1383/2773] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:35 2022] PUT /identity/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9/users/5e8f7c868edf4f939b59bef3f0afb2ce/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:35.947541 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:35.947884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:35.948250 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:35.979235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] MFA Rules not processed for user `5e8f7c868edf4f939b59bef3f0afb2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:35.980505 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:36.055018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1391/2774] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:36.066760 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:36.068842 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:36.094713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:36.139997 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89762-d54c-40ab-afae-8dbd1e81956d admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:36.147282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:36.148715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:36.149147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:36.149629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:36.164406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:36.166783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:36.180856 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1384/2775] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:36.231609 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:36.233901 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:36.250481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067cf8c-d672-4fc0-9c4f-dc822df7f460 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:36.254894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:36.255436 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:36.255702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:36.255979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:36.262811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:36.329242 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:36.330746 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598c22f-3b9b-45a3-b218-3d33c2c1c607 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:36.340703 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1392/2776] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:41.806246 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:41.821111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:41.821618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:41.821618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:41.840226 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] MFA Rules not processed for user `4e12c4cfe11c4be58991df7173e4533f`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:41.841406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:41.872464 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399331c0-6086-497e-b4ec-150bcd5802b8 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:41.877211 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:41.878131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:41.878445 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:41.878724 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] PATH_INFO: `/v3/users/acd5a0db2752455e82c17d4a8cfe111b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:41.879925 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd5a0db2752455e82c17d4a8cfe111b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:41.881882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:41.898829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1393/2777] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:41.934004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd5a0db2752455e82c17d4a8cfe111b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:41.937303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:41.938782 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:41.946166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd5a0db2752455e82c17d4a8cfe111b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:41.955935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd5a0db2752455e82c17d4a8cfe111b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:41.963771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd5a0db2752455e82c17d4a8cfe111b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:41.964482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-f1d7bedc-9bad-47f7-8750-3387c9de37d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd5a0db2752455e82c17d4a8cfe111b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:41.974185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1385/2778] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:41 2022] DELETE /identity/v3/users/acd5a0db2752455e82c17d4a8cfe111b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:41.976974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:41.977268 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:41.977550 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:41.996055 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:41.997342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:41.998679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7de030-e5b9-40ed-ba48-cf730b1007a6 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:42.037260 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:42.048057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcab788fe9b0499c99336de1c06c885d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:42.048855 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:42.049200 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:42.049485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:42.051435 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:42.061812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1386/2779] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:42.072040 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:42.073496 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:42.075955 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:42.129744 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39097241-c96f-4709-9f3f-d0d82befdd24 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:42.134380 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:42.135070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:42.135383 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:42.135649 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:42.144940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:42.146391 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:42.147586 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:42.149063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:42.155933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1387/2780] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:29:42.170131 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1394/2781] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:29:42.556236 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:42.557770 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:42.571870 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24150551-d100-45a4-ae78-eafc2dd261f0 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:42.579097 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:42.580336 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:42.580847 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:42.581023 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] PATH_INFO: `/v3/projects/41f84fc7c5644b438a9271a9c4b66edf` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:42.582416 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f84fc7c5644b438a9271a9c4b66edf)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:42.584414 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:42.647372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f84fc7c5644b438a9271a9c4b66edf', 'request_id': 'req-63e19546-7b83-4fab-b1b3-847939ef5464'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.654317 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f84fc7c5644b438a9271a9c4b66edf', 'request_id': 'req-63e19546-7b83-4fab-b1b3-847939ef5464'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.660711 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f84fc7c5644b438a9271a9c4b66edf', 'request_id': 'req-63e19546-7b83-4fab-b1b3-847939ef5464'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.675095 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] The token cache is being invalidate because project 41f84fc7c5644b438a9271a9c4b66edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:42.675384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.676026 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.677619 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1388/2782] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:42 2022] DELETE /identity/v3/projects/41f84fc7c5644b438a9271a9c4b66edf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:42.681695 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:42.692203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f34f-48ad-4e57-ad3a-188c95e6c4be service glance] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:42.750269 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:42.751469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:42.751594 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:42.751843 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] PATH_INFO: `/v3/users/5e8f7c868edf4f939b59bef3f0afb2ce` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:42.753194 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8f7c868edf4f939b59bef3f0afb2ce)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:42.755452 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:42.810516 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8f7c868edf4f939b59bef3f0afb2ce'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.811753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:42.814021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:42.822715 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8f7c868edf4f939b59bef3f0afb2ce'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.823796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8f7c868edf4f939b59bef3f0afb2ce'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.832111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8f7c868edf4f939b59bef3f0afb2ce'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.850751 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8f7c868edf4f939b59bef3f0afb2ce'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:42.858363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1395/2783] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:42 2022] DELETE /identity/v3/users/5e8f7c868edf4f939b59bef3f0afb2ce => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:42.882968 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19546-7b83-4fab-b1b3-847939ef5464 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:42.915582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:42.951468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:42.952302 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:42.952596 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:42.952876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:42.954853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:43.047628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.049262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325ccf-521f-4c31-a19b-33a6ec101ecb service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.063413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1389/2784] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:29:43.190042 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.192059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:43.200170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:43.273252 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bce13-58bd-4140-9ba8-e62ed64aa241 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.277998 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.278914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.279198 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.279477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] PATH_INFO: `/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.280700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac5edbd6d824a8a9ba08303f4f34ad9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.282641 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.330555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac5edbd6d824a8a9ba08303f4f34ad9', 'request_id': 'req-2681eff8-7f0b-4fbf-bc41-7cac766522dd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:43.337248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac5edbd6d824a8a9ba08303f4f34ad9', 'request_id': 'req-2681eff8-7f0b-4fbf-bc41-7cac766522dd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:43.346959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac5edbd6d824a8a9ba08303f4f34ad9', 'request_id': 'req-2681eff8-7f0b-4fbf-bc41-7cac766522dd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:43.358354 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] The token cache is being invalidate because project 9ac5edbd6d824a8a9ba08303f4f34ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:43.358673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:43.359330 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-2681eff8-7f0b-4fbf-bc41-7cac766522dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:43.360893 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1396/2785] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:43 2022] DELETE /identity/v3/projects/9ac5edbd6d824a8a9ba08303f4f34ad9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:43.365964 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.366262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.366570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.385607 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:43.387185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:43.450600 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1390/2786] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:43.458451 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.458748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.459074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.477275 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:43.478555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:43.486523 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1397/2787] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:43.490769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.500713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43297bb-12a9-4db1-871b-c550ef6738d9 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.505352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.506014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.506295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.506610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.507344 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:29:43.507974 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.509594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.521232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1391/2788] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:29:43.525470 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.538717 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c999ee-616d-4714-899e-dc0b84b72688 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.546438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.547391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.547733 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.548263 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.549277 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.552216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.581459 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1398/2789] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:29:43.586166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.597405 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b8f6-1f60-487d-8b3e-a93d525fe9d8 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.647999 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:43.658400 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.659077 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.659378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.659651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.660345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.662291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.689069 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1392/2790] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:43.690666 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.702545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783babca-2d30-4190-9e8a-9f4d057e61cf admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.707574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.708283 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.708534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.708807 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.709573 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.711215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.722582 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1399/2791] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:43.727245 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.738357 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb57c5-046b-4ebb-be41-cb380c7071fc admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.743107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.743781 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.744066 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.744345 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.745135 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.746780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8232c-95ee-4f04-96c9-3be1f7cb6f49 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.762977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1393/2792] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:43.763747 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.774313 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb19e-b0de-41c3-a66c-589da70f973e admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:43.779634 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:43.780558 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.780840 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.781117 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] PATH_INFO: `/v3/projects/d008f74e022040fd92910b631e59fce3/users/09d33d6e46d943119033c8d7d2a41943/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.801767 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d008f74e022040fd92910b631e59fce3, user_id=09d33d6e46d943119033c8d7d2a41943, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:43.804534 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:43.821458 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1400/2793] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:43 2022] PUT /identity/v3/projects/d008f74e022040fd92910b631e59fce3/users/09d33d6e46d943119033c8d7d2a41943/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:29:43.832258 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:43.832804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:43.832804 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:43.861761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] MFA Rules not processed for user `09d33d6e46d943119033c8d7d2a41943`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:43.863030 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:43.933365 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1394/2794] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:43.938011 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:43.971637 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:44.006486 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185163b-1483-4ae5-9a38-f4af1e93c023 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.011898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.012482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.012763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.013039 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.013749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.015677 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.045749 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1401/2795] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:43 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:44.047034 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.061978 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ada36-ff6c-4711-941a-5b2b48eae530 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.121038 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.121629 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.122483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.122483 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.122914 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.124869 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.150560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1395/2796] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:44 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:44.154829 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.166959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31c348-7cde-4de7-98bd-6e4e67da63e4 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.172151 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.172835 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.173118 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.173395 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.174136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.175801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.195143 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1402/2797] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:44.196056 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.207110 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc22c8-d2b4-43a4-b93a-f3338b2e3f12 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.212065 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.213072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.214053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.214053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] PATH_INFO: `/v3/projects/639d7991057a4153aae92a48a23fd6e3/users/87b8d6614220444bbdf13f3179f9d8b1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.238266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] RBAC: Authorizing `identity:create_grant(project_id=639d7991057a4153aae92a48a23fd6e3, user_id=87b8d6614220444bbdf13f3179f9d8b1, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.238266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.248696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1396/2798] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:44 2022] PUT /identity/v3/projects/639d7991057a4153aae92a48a23fd6e3/users/87b8d6614220444bbdf13f3179f9d8b1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:29:44.252575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.263127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016fecb-3b6a-4caa-a449-fb058ec05ab0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.320921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.321700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.321987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.322270 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.323309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.325109 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f139ff6-32a0-4e37-aab1-01444a3c44a2 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.337213 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1403/2799] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:44.341564 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.351962 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc407-1753-4a41-95b7-47feb759767e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.356780 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.357680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.357965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.358301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] PATH_INFO: `/v3/projects/639d7991057a4153aae92a48a23fd6e3/users/87b8d6614220444bbdf13f3179f9d8b1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.360333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] RBAC: Authorizing `identity:create_grant(project_id=639d7991057a4153aae92a48a23fd6e3, user_id=87b8d6614220444bbdf13f3179f9d8b1, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.363021 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.375296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1397/2800] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:44 2022] PUT /identity/v3/projects/639d7991057a4153aae92a48a23fd6e3/users/87b8d6614220444bbdf13f3179f9d8b1/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:44.386995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.387291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.387575 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.414809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] MFA Rules not processed for user `87b8d6614220444bbdf13f3179f9d8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:44.416181 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:44.486597 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1404/2801] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:29:44.498291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.500284 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:44.520604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:44.588638 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d977e-f272-4b9f-8e19-a2f0c55e967f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.592508 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.593172 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.593461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.593735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.604062 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.605713 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.616144 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1398/2802] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:44.717513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:44.728182 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2aa83-b78b-49c6-8647-869691278dba None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:44.733755 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:44.734530 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:44.734884 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:44.735230 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:44.757685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:44.795794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:44.797256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccddbd-93cb-481c-9bff-8db7f0b9243f service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:44.806460 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1405/2803] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:47.417295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:47.498696 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55c263-a76b-4772-8975-3c250e07a346 service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:47.506894 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:47.508468 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:47.508817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:47.509160 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] PATH_INFO: `/v3/users/09d33d6e46d943119033c8d7d2a41943` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:47.510853 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d33d6e46d943119033c8d7d2a41943)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:47.513482 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:47.565725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d33d6e46d943119033c8d7d2a41943'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:47.578716 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d33d6e46d943119033c8d7d2a41943'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:47.587824 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d33d6e46d943119033c8d7d2a41943'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:47.596299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d33d6e46d943119033c8d7d2a41943'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:47.597054 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d33d6e46d943119033c8d7d2a41943'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:47.603917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1399/2804] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:47 2022] DELETE /identity/v3/users/09d33d6e46d943119033c8d7d2a41943 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:47.609004 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:47.609531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:47.609632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:47.627409 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:47.629255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:47.694919 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1406/2805] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:47.705044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:47.706612 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:47.725852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:47.769377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1166d-2440-46c5-8521-aea859b37437 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:47.773454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:47.774083 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:47.774354 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:47.774650 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:47.784982 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:47.787335 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:47.799850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1400/2806] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:48.396254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:48.397894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:48.411790 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3081954-c789-4841-9f75-9d07244cad2b None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:48.417648 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:48.418601 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:48.418978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:48.419332 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] PATH_INFO: `/v3/projects/d008f74e022040fd92910b631e59fce3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:48.420872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d008f74e022040fd92910b631e59fce3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:48.422887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:48.478438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd008f74e022040fd92910b631e59fce3', 'request_id': 'req-f92f9239-bd56-4aaa-b807-923f7a858bad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.486096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd008f74e022040fd92910b631e59fce3', 'request_id': 'req-f92f9239-bd56-4aaa-b807-923f7a858bad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.496604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd008f74e022040fd92910b631e59fce3', 'request_id': 'req-f92f9239-bd56-4aaa-b807-923f7a858bad'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.512990 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] The token cache is being invalidate because project d008f74e022040fd92910b631e59fce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:48.513291 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.514059 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.516501 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1407/2807] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:48 2022] DELETE /identity/v3/projects/d008f74e022040fd92910b631e59fce3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:48.520903 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:48.531313 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4b08f-0bb0-4fbd-8b6b-37bc6b2b2c96 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:48.596777 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:48.597712 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:48.597989 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:48.598262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] PATH_INFO: `/v3/users/87b8d6614220444bbdf13f3179f9d8b1` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:48.599544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b8d6614220444bbdf13f3179f9d8b1)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:48.601527 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:48.663628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b8d6614220444bbdf13f3179f9d8b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.677180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b8d6614220444bbdf13f3179f9d8b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.686897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b8d6614220444bbdf13f3179f9d8b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.695646 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b8d6614220444bbdf13f3179f9d8b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.696456 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-5c926e1d-920c-4f8d-af05-6a98d883d38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b8d6614220444bbdf13f3179f9d8b1'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:48.704715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1401/2808] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:48 2022] DELETE /identity/v3/users/87b8d6614220444bbdf13f3179f9d8b1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:49.049513 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.053850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:49.071942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f9239-bd56-4aaa-b807-923f7a858bad admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.118927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:49.132727 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.133621 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.133963 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.134398 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] PATH_INFO: `/v3/projects/639d7991057a4153aae92a48a23fd6e3` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.136204 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=639d7991057a4153aae92a48a23fd6e3)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.138227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.188035 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639d7991057a4153aae92a48a23fd6e3', 'request_id': 'req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:49.194349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639d7991057a4153aae92a48a23fd6e3', 'request_id': 'req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:49.203834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639d7991057a4153aae92a48a23fd6e3', 'request_id': 'req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:49.216161 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] The token cache is being invalidate because project 639d7991057a4153aae92a48a23fd6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:49.216492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:49.217222 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-e6f98126-a11e-4dd7-9e1c-6d299d1f2db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:49.219159 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1408/2809] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:49 2022] DELETE /identity/v3/projects/639d7991057a4153aae92a48a23fd6e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:49.224143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.224473 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.224750 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.243427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:49.244719 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:49.278570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:49.306841 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1402/2810] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:49.314694 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.314986 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.315261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.338002 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:49.339889 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:49.349683 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1409/2811] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:49.355643 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.366192 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b4908-28ab-4827-8e5a-2b4368c30761 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.372325 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.372939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.373224 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.373481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.374209 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:29:49.374566 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.376167 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.388891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1403/2812] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:29:49.393960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.407045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4deac71-ff35-4aa4-a994-21fbf5f0a854 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.413064 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.413758 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.414041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.414309 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.415050 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.416985 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.448333 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1410/2813] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:49.452146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.462683 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500956e8-569e-462b-a2d2-de6f65f5b2d1 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.522319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.523001 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.523287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.523561 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.524257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.526191 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.547941 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1404/2814] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:29:49.554590 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.566772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603fc54-cb10-4519-b9ec-41cbe8fa9889 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.572190 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.572861 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.573145 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.573427 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.574167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.575959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.586489 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1411/2815] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:49.590855 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.600867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96d321-7ea4-489f-b6cd-9bf1fdc6e63a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.605570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.606225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.606542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.606819 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.607528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.609124 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6526247-93ef-441c-837c-b22367384396 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.619068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1405/2816] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:49.623280 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.634506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5080e7-6860-47c8-a2c6-12e644d4d1c9 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.639918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.640821 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.641108 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.641382 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] PATH_INFO: `/v3/projects/316e27d749d54849ac42b4375d6012fd/users/d437500b25d740d69120cd69c3e0fd4f/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.659877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316e27d749d54849ac42b4375d6012fd, user_id=d437500b25d740d69120cd69c3e0fd4f, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.663636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.679293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1412/2817] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:49 2022] PUT /identity/v3/projects/316e27d749d54849ac42b4375d6012fd/users/d437500b25d740d69120cd69c3e0fd4f/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:49.758769 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.759061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.759342 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.787180 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] MFA Rules not processed for user `d437500b25d740d69120cd69c3e0fd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:49.788395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:49.857868 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1406/2818] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:49.862981 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.930359 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728f2ba-023f-4608-be22-f55ee0d13255 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:49.935809 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:49.936495 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:49.936768 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:49.937045 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:49.937792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:49.939738 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:49.961577 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:49.969819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1413/2819] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:49.973977 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:49.985296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26e0c-5537-4ebf-94d4-2b184d3b6bd1 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.024372 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:50.061422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.062136 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.062429 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.062739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.063453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.065387 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.093301 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1407/2820] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:49 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:50.097434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.112995 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a3c-bbec-4a08-95d8-8bf9cdb11451 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.119061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.119791 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.120178 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.120482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.121443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.123140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.134636 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1414/2821] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:50.139687 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.150227 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73fcf6-6d04-40ae-8a3a-20f66d4f0d7e admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.155665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.156594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.156880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.157157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] PATH_INFO: `/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c/users/333cc4b91f604c2195ea57cc7739645b/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.173133 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9f6f6fa30a4c48b9d75874d5aeb54c, user_id=333cc4b91f604c2195ea57cc7739645b, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.175863 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.191563 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1408/2822] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:50 2022] PUT /identity/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c/users/333cc4b91f604c2195ea57cc7739645b/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:50.195574 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.206389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b54f5-5d55-4f18-b8dc-565772ba66c8 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.267138 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.267933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.268274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.268607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.269408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.271102 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79ea38-5764-4db6-a50f-72d018f26732 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.282631 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1415/2823] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:50.286720 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.297557 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02765f-bad3-4811-bdf9-be4ce436e141 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.302244 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.303277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.303660 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.304031 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] PATH_INFO: `/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c/users/333cc4b91f604c2195ea57cc7739645b/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.306107 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9f6f6fa30a4c48b9d75874d5aeb54c, user_id=333cc4b91f604c2195ea57cc7739645b, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.308971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.320663 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1409/2824] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:50 2022] PUT /identity/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c/users/333cc4b91f604c2195ea57cc7739645b/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:29:50.331699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.331699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.331699 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.364692 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] MFA Rules not processed for user `333cc4b91f604c2195ea57cc7739645b`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:50.366027 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:50.452139 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1416/2825] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:50.463319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.465273 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:50.522151 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707820-3dfc-4956-8351-aad3435630a0 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.526215 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.527078 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.527279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.527728 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.537433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.538928 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.549222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1410/2826] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:50.636923 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:50.647862 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0656-8ef3-4445-a606-7773d2592e47 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:50.652403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:50.653061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:50.653341 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:50.653642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:50.712292 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:50.713811 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f391373-1bc3-462e-94bd-1a4e32f1bef0 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:50.723105 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1417/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:55.720430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:55.803286 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1e03-7500-45d4-99b4-3011c2052d5b service cinder] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:55.808485 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:55.809390 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:55.809665 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:55.809939 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] PATH_INFO: `/v3/users/d437500b25d740d69120cd69c3e0fd4f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:55.811185 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d437500b25d740d69120cd69c3e0fd4f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:55.813162 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:55.850782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:55.865528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd437500b25d740d69120cd69c3e0fd4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:55.877381 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd437500b25d740d69120cd69c3e0fd4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:55.886682 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd437500b25d740d69120cd69c3e0fd4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:55.894079 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd437500b25d740d69120cd69c3e0fd4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:55.894831 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd437500b25d740d69120cd69c3e0fd4f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:55.900806 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1411/2828] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:55 2022] DELETE /identity/v3/users/d437500b25d740d69120cd69c3e0fd4f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:55.905545 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:55.906492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:55.906492 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:55.918308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=100 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:55.925020 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:55.926239 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:55.996180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1418/2829] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:56.006507 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:56.008384 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:56.072745 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2626f-b124-4b27-a1fe-5a9dbb9f9a00 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:56.077755 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:56.079562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:56.079844 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:56.080131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:56.090933 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:56.092479 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:56.107143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1412/2830] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:56.387965 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:56.389560 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:56.402074 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ceee8-0b70-4e9e-90ed-6ed6a546bf20 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:56.406969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:56.407857 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:56.408142 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:56.408415 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] PATH_INFO: `/v3/projects/316e27d749d54849ac42b4375d6012fd` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:56.409599 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=316e27d749d54849ac42b4375d6012fd)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:56.411555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:56.460953 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316e27d749d54849ac42b4375d6012fd', 'request_id': 'req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.467860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316e27d749d54849ac42b4375d6012fd', 'request_id': 'req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.477369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316e27d749d54849ac42b4375d6012fd', 'request_id': 'req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.487568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] The token cache is being invalidate because project 316e27d749d54849ac42b4375d6012fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:56.487859 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.488482 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.490094 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1419/2831] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:56 2022] DELETE /identity/v3/projects/316e27d749d54849ac42b4375d6012fd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:56.494177 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:56.504257 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4e0e5-2e8a-472e-8443-dc720704cd92 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:56.565359 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:56.566254 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:56.566559 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:56.566833 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] PATH_INFO: `/v3/users/333cc4b91f604c2195ea57cc7739645b` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:56.568164 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333cc4b91f604c2195ea57cc7739645b)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:56.570241 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:56.621664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333cc4b91f604c2195ea57cc7739645b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.626183 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:56.636266 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333cc4b91f604c2195ea57cc7739645b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.645202 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333cc4b91f604c2195ea57cc7739645b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.652053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333cc4b91f604c2195ea57cc7739645b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.652761 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-d25e1626-3f71-418e-991a-29a918819489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333cc4b91f604c2195ea57cc7739645b'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:56.658783 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1413/2832] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:56 2022] DELETE /identity/v3/users/333cc4b91f604c2195ea57cc7739645b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:29:56.950325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:56.952000 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:56.965236 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286dd6d-b0ae-4b86-ab5d-396c36f01aee admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.024787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.025663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.025943 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.026214 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] PATH_INFO: `/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.027429 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9f6f6fa30a4c48b9d75874d5aeb54c)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.029329 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.068350 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:57.082062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9f6f6fa30a4c48b9d75874d5aeb54c', 'request_id': 'req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:57.088238 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9f6f6fa30a4c48b9d75874d5aeb54c', 'request_id': 'req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:57.096287 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9f6f6fa30a4c48b9d75874d5aeb54c', 'request_id': 'req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:57.106254 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] The token cache is being invalidate because project 9f9f6f6fa30a4c48b9d75874d5aeb54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:57.106581 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:57.107189 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-08dd87d7-a5f1-4ef8-adea-d8d7135cd41b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:57.108804 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1420/2833] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:56 2022] DELETE /identity/v3/projects/9f9f6f6fa30a4c48b9d75874d5aeb54c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:57.113767 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.114292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.114292 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.132956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:57.134187 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:57.199714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1414/2834] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:57.207711 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.207999 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.208271 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.228010 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:57.229202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:57.237548 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1421/2835] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:57.242554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.255196 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973718a9-a166-4377-97c1-e8f7e4a3a327 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.261598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.262281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.262603 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.262880 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.263652 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:29:57.263961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.265814 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.278353 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1415/2836] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:29:57.283080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.293124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b84e5-b5ad-45a1-b24d-c65db236eefa None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.299049 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.299704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.299982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.300261 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.300961 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.302908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.331863 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1422/2837] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:57.336182 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.346931 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29304b-d0e3-4ee4-9343-2cb3944f5b85 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.406972 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.407651 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.407934 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.408212 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.408910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.410878 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.431368 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1416/2838] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:57.436886 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.447606 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f57aa-cd64-421c-9558-7de3074e88b5 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.452469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.453134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.453419 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.453698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.454467 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.456070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.466913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1423/2839] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:57.471080 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.481385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b6259-6e5c-444a-b6aa-e11a1aa7e18a admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.487190 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.488166 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.488622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.489035 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.490147 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.492876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc77a5-d58b-434c-8d30-16b7fe8d6243 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.504061 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1417/2840] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:57.511089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.528655 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728492a8-7256-4e3a-a8b8-fb29b3c21533 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.535796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.536674 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.536978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.537257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] PATH_INFO: `/v3/projects/629bb7fd441145ff9564a89ca9cb7f67/users/9438b5ba2c5b4dd3bfd2ecc9228f9ee9/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.554025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629bb7fd441145ff9564a89ca9cb7f67, user_id=9438b5ba2c5b4dd3bfd2ecc9228f9ee9, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.556718 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.573403 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1424/2841] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:57 2022] PUT /identity/v3/projects/629bb7fd441145ff9564a89ca9cb7f67/users/9438b5ba2c5b4dd3bfd2ecc9228f9ee9/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:57.585221 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.585542 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.585823 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.613002 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] MFA Rules not processed for user `9438b5ba2c5b4dd3bfd2ecc9228f9ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:57.614114 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:57.647006 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:57.689965 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1418/2842] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:57.694709 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.760704 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c5b8-894d-4233-8260-9a940ecaca12 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.765834 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.766538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.766822 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.767096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.767816 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.769765 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.798441 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1425/2843] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:57.802776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.813920 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e3ee7-f2c3-4777-bfc6-bbff118c5403 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.871470 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.872064 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.872349 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.872624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.873319 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.875543 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.898431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1419/2844] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:29:57 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:29:57.902593 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.913642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244473d7-ece8-4e20-97f0-78e2c5d5e107 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.918744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.919411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.919702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.919979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.920719 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.922362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.933153 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1426/2845] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:57.937421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:57.948157 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc32bde-4002-4cef-96c4-5a2d2f18a698 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:57.954174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:57.955090 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:57.955369 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:57.955644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] PATH_INFO: `/v3/projects/0708926634c5457e9d3c485a9b0685cc/users/dae3dce01dd64698890ea14847da41d8/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:57.971919 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0708926634c5457e9d3c485a9b0685cc, user_id=dae3dce01dd64698890ea14847da41d8, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:57.974636 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:57.990714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1420/2846] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:57 2022] PUT /identity/v3/projects/0708926634c5457e9d3c485a9b0685cc/users/dae3dce01dd64698890ea14847da41d8/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 04:29:57.994819 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:58.004872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944db0-a0e8-4a81-bd41-890dfb674e38 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:58.033389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:58.064018 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:58.064685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.064969 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.065248 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.065978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:58.067691 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b656a9-0445-482b-8b1d-e8a027327209 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:58.078286 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1427/2847] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:29:58.082730 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:58.093858 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253d16-61b0-45c2-8e8b-55d1524c5fd6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:58.098785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:58.099678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.099961 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.102220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] PATH_INFO: `/v3/projects/0708926634c5457e9d3c485a9b0685cc/users/dae3dce01dd64698890ea14847da41d8/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.104333 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0708926634c5457e9d3c485a9b0685cc, user_id=dae3dce01dd64698890ea14847da41d8, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:58.107049 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:58.119631 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1421/2848] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:29:58 2022] PUT /identity/v3/projects/0708926634c5457e9d3c485a9b0685cc/users/dae3dce01dd64698890ea14847da41d8/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:58.130133 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.130363 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.130685 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.160602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] MFA Rules not processed for user `dae3dce01dd64698890ea14847da41d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:58.161844 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:58.234262 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1428/2849] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:58.245222 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:58.247279 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:58.306296 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a19c0-f7f0-47ba-934d-de5859bddca5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:58.310339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:58.311014 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.311262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.311540 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.321275 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:58.322751 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32208d-233b-41f8-90aa-2b3c4de27406 service cinder] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:58.332422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1422/2850] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:58.715604 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:58.779933 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88529fcc-2627-4bfb-86a4-efdc68bca3ed None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:58.785772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:58.786702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.786984 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.787293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] PATH_INFO: `/v3/users/9438b5ba2c5b4dd3bfd2ecc9228f9ee9` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.788499 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9438b5ba2c5b4dd3bfd2ecc9228f9ee9)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:58.790555 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:58.839304 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9438b5ba2c5b4dd3bfd2ecc9228f9ee9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:58.848302 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9438b5ba2c5b4dd3bfd2ecc9228f9ee9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:58.848908 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9438b5ba2c5b4dd3bfd2ecc9228f9ee9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:58.850256 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:58.858057 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9438b5ba2c5b4dd3bfd2ecc9228f9ee9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:58.869959 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9438b5ba2c5b4dd3bfd2ecc9228f9ee9'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:58.876673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1429/2851] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:58 2022] DELETE /identity/v3/users/9438b5ba2c5b4dd3bfd2ecc9228f9ee9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:58.881560 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:58.881850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:58.882127 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:58.902554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:29:58.903905 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:58.924189 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:58.977943 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1423/2852] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:29:58.989630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:58.991517 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:59.050979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f07cff-48be-4e47-83e9-2cfd89701903 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:59.055511 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:59.056171 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:59.056472 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:59.056744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:59.071617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:59.073079 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:59.083028 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1430/2853] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:29:59.367586 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:59.369216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:59.383451 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c0844-e52e-4ab9-a018-dc87ad713291 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:59.388386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:59.389476 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:59.389622 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:59.389872 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] PATH_INFO: `/v3/projects/629bb7fd441145ff9564a89ca9cb7f67` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:59.391762 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629bb7fd441145ff9564a89ca9cb7f67)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:59.393705 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:59.444747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629bb7fd441145ff9564a89ca9cb7f67', 'request_id': 'req-485de542-86dc-41fa-9d05-626367ebdbf5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.450867 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629bb7fd441145ff9564a89ca9cb7f67', 'request_id': 'req-485de542-86dc-41fa-9d05-626367ebdbf5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.456602 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629bb7fd441145ff9564a89ca9cb7f67', 'request_id': 'req-485de542-86dc-41fa-9d05-626367ebdbf5'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.471291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] The token cache is being invalidate because project 629bb7fd441145ff9564a89ca9cb7f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:29:59.471582 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.472229 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.473815 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1424/2854] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:59 2022] DELETE /identity/v3/projects/629bb7fd441145ff9564a89ca9cb7f67 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:59.477957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:59.488184 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1ca64-a3fa-497e-8e19-c9e7fd8c73dc service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:29:59.546918 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:29:59.547800 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:29:59.548084 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:29:59.548362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] PATH_INFO: `/v3/users/dae3dce01dd64698890ea14847da41d8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:29:59.549578 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae3dce01dd64698890ea14847da41d8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:29:59.551546 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:29:59.602358 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae3dce01dd64698890ea14847da41d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.611554 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae3dce01dd64698890ea14847da41d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.612197 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae3dce01dd64698890ea14847da41d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.619165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae3dce01dd64698890ea14847da41d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.629799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:29:59.632852 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-c671c085-236c-4eb0-92e0-ddc8895efb64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae3dce01dd64698890ea14847da41d8'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:29:59.640656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1431/2855] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:29:59 2022] DELETE /identity/v3/users/dae3dce01dd64698890ea14847da41d8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:29:59.926877 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:29:59.928402 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:29:59.942010 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485de542-86dc-41fa-9d05-626367ebdbf5 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.000459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.001352 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.001627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.001897 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] PATH_INFO: `/v3/projects/0708926634c5457e9d3c485a9b0685cc` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.003116 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0708926634c5457e9d3c485a9b0685cc)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.005027 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.051441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:00.054525 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0708926634c5457e9d3c485a9b0685cc', 'request_id': 'req-2d324227-75f1-479c-8530-80e6f1b1ee06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:00.061218 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0708926634c5457e9d3c485a9b0685cc', 'request_id': 'req-2d324227-75f1-479c-8530-80e6f1b1ee06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:00.067672 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0708926634c5457e9d3c485a9b0685cc', 'request_id': 'req-2d324227-75f1-479c-8530-80e6f1b1ee06'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:00.082099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] The token cache is being invalidate because project 0708926634c5457e9d3c485a9b0685cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:30:00.082413 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:00.083041 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-2d324227-75f1-479c-8530-80e6f1b1ee06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:00.084610 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1425/2856] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:29:59 2022] DELETE /identity/v3/projects/0708926634c5457e9d3c485a9b0685cc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:00.089833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.090308 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.090450 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.108404 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:00.109882 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:00.173888 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1432/2857] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:00.183194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.183472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.183772 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.203427 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:00.204748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:00.214281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1426/2858] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:00.218774 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.229720 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e46c5-eac9-4073-b46e-006ac154699a None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.235408 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.236067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.236357 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.236632 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.237369 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:30:00.237675 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.239314 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.252140 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1433/2859] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:30:00.256431 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.266594 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866056c9-4cf0-48b2-b1d8-b14ce2b9e6f5 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.272812 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.273466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.273747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.274015 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.274752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.276644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.307752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1427/2860] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:30:00.311272 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.321455 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab053c-ce33-4043-8644-016b0b483a41 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.390914 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.391910 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.392366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.392913 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.393658 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.396616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.422960 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1434/2861] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:30:00.427242 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.438099 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11bf1-c05d-4c90-accb-76aad7c0bf26 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.443620 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.444285 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.444568 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.444834 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.445575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.447220 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.458146 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1428/2862] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:00.462054 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.472532 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729007-8ea0-4dba-a8bd-45af01529d74 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.478654 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.479320 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.479602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.479876 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.480614 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.482196 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc505d1-da03-4c92-892a-d60b0fe7e4ec admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.492978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1435/2863] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:00.498331 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.509580 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57ba73-752a-42b1-a946-fd2a9ef28a53 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.515278 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.516174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.516528 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.516807 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] PATH_INFO: `/v3/projects/2e28f4701c034654b59bce927e7be1f8/users/dd143626a82f4f0f8af7486b472c7873/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.531771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e28f4701c034654b59bce927e7be1f8, user_id=dd143626a82f4f0f8af7486b472c7873, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.534465 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.549739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1429/2864] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:30:00 2022] PUT /identity/v3/projects/2e28f4701c034654b59bce927e7be1f8/users/dd143626a82f4f0f8af7486b472c7873/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:00.560060 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.560349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.560630 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.587714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] MFA Rules not processed for user `dd143626a82f4f0f8af7486b472c7873`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:00.589062 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:00.639526 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:00.660912 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1436/2865] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:00.665481 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.733671 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf31ac-c094-49fd-a584-b23a0951b378 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.739644 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.740314 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.740598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.740875 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.741572 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.743532 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.772710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1430/2866] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:30:00.777538 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.790833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ff0e3-dc2d-4ecd-9335-fd53cff56201 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.849841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.850769 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.851061 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.851348 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] PATH_INFO: `/v3/users` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.852067 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.853987 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.875602 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1437/2867] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:30:00 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:30:00.879628 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.890735 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a19f9-8894-40c3-9db4-11ed65b075fe admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.895885 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.896575 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.896852 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.897118 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.897851 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.899509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.910598 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1431/2868] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:00.914799 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.925186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929528e-f3f7-4a4c-b2eb-e8feeeb08425 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:00.930434 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:00.931319 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:00.931603 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:00.931875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] PATH_INFO: `/v3/projects/6c5b60c779264a30a99ececdb5544ba4/users/aec9d5c56de14685aed15902e93183ee/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:00.947702 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5b60c779264a30a99ececdb5544ba4, user_id=aec9d5c56de14685aed15902e93183ee, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:00.950464 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:00.967037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1438/2869] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:30:00 2022] PUT /identity/v3/projects/6c5b60c779264a30a99ececdb5544ba4/users/aec9d5c56de14685aed15902e93183ee/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:00.971232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:00.981216 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55876c-6ac4-4f8d-b79e-d213b7f85e6f admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:01.022839 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:01.042461 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:01.043109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:01.043396 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:01.043679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:01.044420 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:01.046070 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72544a-70e0-4c4a-b06e-3f65a17604ed admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:01.056109 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1432/2870] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:01.060406 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:01.071098 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc866f-92e1-46ca-947e-29c68c490f02 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:01.077041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:01.078016 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:01.078303 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:01.078619 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] PATH_INFO: `/v3/projects/6c5b60c779264a30a99ececdb5544ba4/users/aec9d5c56de14685aed15902e93183ee/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:01.080865 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5b60c779264a30a99ececdb5544ba4, user_id=aec9d5c56de14685aed15902e93183ee, role_id=ded6d8407dfa44b5bf6c02ed2c0bcd40)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:01.083763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:01.094927 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1439/2871] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:30:01 2022] PUT /identity/v3/projects/6c5b60c779264a30a99ececdb5544ba4/users/aec9d5c56de14685aed15902e93183ee/roles/ded6d8407dfa44b5bf6c02ed2c0bcd40 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:01.105228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:01.105556 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:01.105876 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:01.134287 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] MFA Rules not processed for user `aec9d5c56de14685aed15902e93183ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:01.135546 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:01.207073 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1433/2872] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:01.218089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:01.220296 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:01.287983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c68a0-576c-4f07-bff1-a6444475d14f admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:01.291957 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:01.292611 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:01.292898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:01.293179 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:01.359411 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:01.362063 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:01.363679 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eec2d6-8d0e-4e6a-be90-45a4268d4d7d service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:01.373979 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1440/2873] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:03.947654 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:04.024926 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986fad8-2835-411c-9f92-704be81d1a52 None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:04.032896 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:04.033799 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:04.034081 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:04.034398 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] PATH_INFO: `/v3/users/dd143626a82f4f0f8af7486b472c7873` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:04.035700 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd143626a82f4f0f8af7486b472c7873)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:04.037664 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:04.092150 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd143626a82f4f0f8af7486b472c7873'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.104995 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd143626a82f4f0f8af7486b472c7873'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.107657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:04.114277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd143626a82f4f0f8af7486b472c7873'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.123072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd143626a82f4f0f8af7486b472c7873'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.123715 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd143626a82f4f0f8af7486b472c7873'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.130326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1434/2874] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:30:03 2022] DELETE /identity/v3/users/dd143626a82f4f0f8af7486b472c7873 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:04.135203 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:04.135722 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:04.135827 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:04.153714 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:04.154989 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:04.219713 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1441/2875] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:30:04.229895 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:04.231520 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:04.305385 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984447-8916-4b52-bc59-b9bd230008e6 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:04.310722 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:04.311375 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:04.311662 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:04.311927 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:04.324956 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:04.326415 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:04.336589 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1435/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:04.646484 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:04.647970 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:04.662124 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d6936-4fe4-4a29-b0dc-43e968425474 None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:04.667942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:04.669519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:04.669806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:04.670078 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] PATH_INFO: `/v3/projects/2e28f4701c034654b59bce927e7be1f8` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:04.671592 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e28f4701c034654b59bce927e7be1f8)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:04.673616 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:04.722735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e28f4701c034654b59bce927e7be1f8', 'request_id': 'req-700c2dcb-09c8-4af9-b27c-f6d267250e25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.728817 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e28f4701c034654b59bce927e7be1f8', 'request_id': 'req-700c2dcb-09c8-4af9-b27c-f6d267250e25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.737684 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e28f4701c034654b59bce927e7be1f8', 'request_id': 'req-700c2dcb-09c8-4af9-b27c-f6d267250e25'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.748325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] The token cache is being invalidate because project 2e28f4701c034654b59bce927e7be1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:30:04.748613 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.749260 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.751013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1442/2877] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:30:04 2022] DELETE /identity/v3/projects/2e28f4701c034654b59bce927e7be1f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:04.755240 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:04.765693 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d47fea-ddb5-479f-8529-a3af97a4d433 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:04.831036 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:04.831913 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:04.832239 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:04.832509 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] PATH_INFO: `/v3/users/aec9d5c56de14685aed15902e93183ee` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:04.833714 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec9d5c56de14685aed15902e93183ee)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:04.835797 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:04.885723 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec9d5c56de14685aed15902e93183ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.897562 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec9d5c56de14685aed15902e93183ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.905626 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec9d5c56de14685aed15902e93183ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.913037 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec9d5c56de14685aed15902e93183ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.913782 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-4ba67899-e47f-4bb4-8e45-627508e506f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec9d5c56de14685aed15902e93183ee'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:04.920424 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1436/2878] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:30:04 2022] DELETE /identity/v3/users/aec9d5c56de14685aed15902e93183ee => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:05.200875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.202553 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:05.216228 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2dcb-09c8-4af9-b27c-f6d267250e25 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.275875 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.276748 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.277025 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.277299 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] PATH_INFO: `/v3/projects/6c5b60c779264a30a99ececdb5544ba4` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.278542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5b60c779264a30a99ececdb5544ba4)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.280485 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.306240 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:05.337850 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5b60c779264a30a99ececdb5544ba4', 'request_id': 'req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:05.344307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5b60c779264a30a99ececdb5544ba4', 'request_id': 'req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:05.354191 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5b60c779264a30a99ececdb5544ba4', 'request_id': 'req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:05.364416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] The token cache is being invalidate because project 6c5b60c779264a30a99ececdb5544ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:30:05.364698 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:05.365325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-72fe4d8e-0ff2-472f-8b3b-12445b3f8767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:05.366924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1443/2879] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:30:05 2022] DELETE /identity/v3/projects/6c5b60c779264a30a99ececdb5544ba4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:05.371917 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.372426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.372533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.385865 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:05.392776 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:05.394891 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:05.468272 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1437/2880] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 04:30:05.477568 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.477858 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.478134 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.498041 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:05.499293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:05.508787 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1444/2881] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:05.513721 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.528738 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c605c-54b1-43c2-a70c-0ccac198905c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.535536 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.536195 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.536477 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.536747 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] PATH_INFO: `/v3/domains` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.537467 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 04:30:05.537771 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.539394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.551453 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1438/2882] 10.222.0.16 () {58 vars in 1152 bytes} [Mon Jul 4 04:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 04:30:05.555993 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.565924 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2be3ec-ebce-4ab0-94a4-313136439f5d None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.572089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.572841 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.573120 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.573425 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.574290 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.576535 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.609502 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1445/2883] 10.222.0.16 () {60 vars in 1152 bytes} [Mon Jul 4 04:30:05 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 04:30:05.614000 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.625882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941aec-bf70-41e8-9a92-1a1b7d4ae47c admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.687458 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.688169 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.688454 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.688739 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] PATH_INFO: `/v3/users` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.689444 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.691394 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.717680 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1439/2884] 10.222.0.16 () {60 vars in 1143 bytes} [Mon Jul 4 04:30:05 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 04:30:05.724430 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.741951 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374be52d-b679-4f76-8ad3-fc67e2e337e0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.750700 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.751375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.751656 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.751925 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.752687 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.754276 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.765724 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1446/2885] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:05.770291 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.781323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054eda8-7c2d-48c6-b3bc-0573519d68eb admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.787382 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.788072 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.788356 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.788627 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.789358 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.791028 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1dc0-bba4-4191-82ed-b8b50bfe3e10 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.802043 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1440/2886] 10.222.0.16 () {58 vars in 1121 bytes} [Mon Jul 4 04:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 04:30:05.806362 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.816475 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04031c45-3d6d-46b3-8a11-2543d50768bd admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:05.821443 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:05.822325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.822627 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.822926 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] PATH_INFO: `/v3/projects/d44e54b708af41df844d44910c1ac5be/users/e630d017dac8449a9d48076c7443169d/roles/d4760ffd4761422abfabd45bc0880415` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.837488 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44e54b708af41df844d44910c1ac5be, user_id=e630d017dac8449a9d48076c7443169d, role_id=d4760ffd4761422abfabd45bc0880415)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:05.840165 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:05.855673 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1447/2887] 10.222.0.16 () {60 vars in 1482 bytes} [Mon Jul 4 04:30:05 2022] PUT /identity/v3/projects/d44e54b708af41df844d44910c1ac5be/users/e630d017dac8449a9d48076c7443169d/roles/d4760ffd4761422abfabd45bc0880415 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:05.866422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] REQUEST_METHOD: `POST` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:05.866748 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:05.867011 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:05.898909 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.auth.core [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] MFA Rules not processed for user `e630d017dac8449a9d48076c7443169d`. Rule list: `[]` (Enabled: `True`). {{(pid=80371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:30:05.900140 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:05.970299 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1441/2888] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:30:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:30:05.981937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:05.984089 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:06.050355 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8560830-3f69-4393-8f52-f197bceffdd0 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:06.054540 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f9c5095b2ac44c1833141d0949c6ef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:06.055206 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:06.055481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:06.055772 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:06.065127 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:06.066638 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:06.076149 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1448/2889] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:15.966835 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:16.071741 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:26.754326 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:26.756022 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:26.820678 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ae144-02ea-45b7-83ab-00caa82f334c None None] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:26.824510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:26.825153 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:26.825441 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:26.825717 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:26.829050 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:26.838277 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:26.839785 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:26.848173 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1442/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:33.514937 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:33.530978 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5106-14ec-4a9b-9ae8-ff4758a3b1c5 service cinder] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:33.535469 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:33.536136 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:33.536416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:33.536688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:33.538794 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:33.549208 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:33.551168 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:33.560298 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1449/2891] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:30:33.663530 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:33.672235 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7c27a-f639-4666-a71d-2350c5ef278b service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:33.678595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:33.678595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:33.678595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:33.678595 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:33.679175 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:33.760466 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:33.761971 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:33.774821 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1443/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:43.556325 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=6 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:43.768987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=60 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:30:53.500942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:53.503162 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:53.557666 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:53.559339 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:53.586021 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c41c-c12b-4020-9c1a-6a418e366809 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:53.590968 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:53.591670 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:53.591952 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:53.592227 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:53.594068 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:30:53.606111 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:53.607710 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:53.617618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1450/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:30:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:30:53.634122 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da452d-fb70-476b-a5b4-000c178b6357 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:53.639635 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:53.640533 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:53.640817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:53.641092 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] PATH_INFO: `/v3/users/4e12c4cfe11c4be58991df7173e4533f` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:53.642281 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e12c4cfe11c4be58991df7173e4533f)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:53.644208 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:53.694361 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e12c4cfe11c4be58991df7173e4533f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:53.708817 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e12c4cfe11c4be58991df7173e4533f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:53.718251 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e12c4cfe11c4be58991df7173e4533f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:53.725706 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e12c4cfe11c4be58991df7173e4533f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:53.726386 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e12c4cfe11c4be58991df7173e4533f'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:53.732886 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1444/2894] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:30:53 2022] DELETE /identity/v3/users/4e12c4cfe11c4be58991df7173e4533f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:30:53.918942 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:30:53.930801 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c2e2-6d47-4db1-bf17-344af294c104 service placement] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:30:53.989972 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:30:53.990898 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:30:53.991186 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:30:53.991477 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] PATH_INFO: `/v3/projects/5cfc2721cf3146beb4471ded02b1b637` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:30:53.992642 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cfc2721cf3146beb4471ded02b1b637)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:30:53.994556 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:30:54.043531 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cfc2721cf3146beb4471ded02b1b637', 'request_id': 'req-11c3dc93-310b-428e-8486-14184208dfca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:54.049837 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cfc2721cf3146beb4471ded02b1b637', 'request_id': 'req-11c3dc93-310b-428e-8486-14184208dfca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:54.058576 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cfc2721cf3146beb4471ded02b1b637', 'request_id': 'req-11c3dc93-310b-428e-8486-14184208dfca'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:54.069282 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] The token cache is being invalidate because project 5cfc2721cf3146beb4471ded02b1b637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:30:54.069571 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:54.070215 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-11c3dc93-310b-428e-8486-14184208dfca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:30:54.071983 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1451/2895] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:30:53 2022] DELETE /identity/v3/projects/5cfc2721cf3146beb4471ded02b1b637 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:31:00.341624 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:31:00.343316 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:31:00.358142 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a88f-1f2c-4d41-9649-87b6fcea7d05 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:31:00.429801 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:31:00.430710 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:31:00.430987 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:31:00.431262 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] PATH_INFO: `/v3/users/e630d017dac8449a9d48076c7443169d` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:31:00.432503 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e630d017dac8449a9d48076c7443169d)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:31:00.434406 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:31:00.456004 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:31:00.488129 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e630d017dac8449a9d48076c7443169d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:00.500583 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e630d017dac8449a9d48076c7443169d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:00.510433 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e630d017dac8449a9d48076c7443169d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:00.518053 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e630d017dac8449a9d48076c7443169d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:00.518765 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e630d017dac8449a9d48076c7443169d'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:00.524645 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1445/2896] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:31:00 2022] DELETE /identity/v3/users/e630d017dac8449a9d48076c7443169d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:31:00.531183 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] REQUEST_METHOD: `POST` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:31:00.531723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:31:00.531723 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:31:00.549527 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.auth.core [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] MFA Rules not processed for user `769a75081be74879911bd8c4f4cb57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 04:31:00.550762 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:31:00.565077 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:31:00.621806 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1452/2897] 10.222.0.16 () {58 vars in 957 bytes} [Mon Jul 4 04:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 04:31:00.632068 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:31:00.633726 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:31:00.695822 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21120-5935-44c3-895a-8ab4f798da74 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:31:00.700225 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:31:00.700893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:31:00.701174 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:31:00.701446 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:31:00.711675 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:31:00.713111 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:31:00.723906 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1446/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:31:01.019805 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:31:01.021293 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:31:01.036997 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c7f42-8c27-4ab3-8c51-7001d22000df None None] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:31:01.041752 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:31:01.042663 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:31:01.042944 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:31:01.043216 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] PATH_INFO: `/v3/projects/d44e54b708af41df844d44910c1ac5be` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:31:01.044438 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44e54b708af41df844d44910c1ac5be)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:31:01.046342 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:31:01.095366 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44e54b708af41df844d44910c1ac5be', 'request_id': 'req-cf0fc017-5d6e-4555-a394-da2a2fcd445c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:01.101391 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44e54b708af41df844d44910c1ac5be', 'request_id': 'req-cf0fc017-5d6e-4555-a394-da2a2fcd445c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:01.112202 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44e54b708af41df844d44910c1ac5be', 'request_id': 'req-cf0fc017-5d6e-4555-a394-da2a2fcd445c'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:01.123563 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] The token cache is being invalidate because project d44e54b708af41df844d44910c1ac5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:31:01.123848 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:01.124506 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:31:01.126096 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1453/2899] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:31:01 2022] DELETE /identity/v3/projects/d44e54b708af41df844d44910c1ac5be => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:31:10.719032 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:DELETE=7,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:31:11.120567 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:26.088725 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:26.098679 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a783-c474-4c94-afb9-9fd267367be7 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:26.151232 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:26.151893 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:26.152197 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:26.152472 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:26.154502 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:26.216044 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:26.217501 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:26.229282 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1447/2900] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:32:27.095258 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:27.096796 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:27.181906 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fc017-5d6e-4555-a394-da2a2fcd445c admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:27.189170 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:27.189784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:27.190070 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:27.190349 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:27.192446 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:27.206519 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:27.208877 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:27.221872 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1454/2901] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 04:32:29.981395 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:29.995749 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64458a2-3944-4da1-8539-9cf85508f314 service neutron] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:30.001459 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789dd1683eac49d9b26609eb2903475e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:30.002108 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:30.002570 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:30.002850 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:30.004843 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:30.018604 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:30.020228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:30.031510 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1448/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:32:32.192033 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:32.193585 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:32.206080 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52983b-9db6-45df-affa-c8cde82a2d9d service nova] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:32.209647 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e468089828e4180b8636e167d16cf8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:32.210255 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:32.210618 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:32.210892 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:32.212583 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:32.289815 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:32.291289 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:32.301123 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1455/2903] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:32:35.423695 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:35.425188 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:35.526753 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc29fd6-f4ed-4c42-a823-4d50fe70cd65 service nova] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:35.531692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:35.532351 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:35.532633 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:35.532910 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:35.534694 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:35.547616 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:35.549554 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:35.560005 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1449/2904] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:32:38.713389 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:38.714920 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:38.728626 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab110a8-a62d-4788-b4ef-f5e6aa98d259 service neutron] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:38.733232 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:38.734135 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:38.734456 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:38.734744 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] PATH_INFO: `/v3/users/b8158ebeedf7438b9e3306186c1898cd` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:38.735921 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8158ebeedf7438b9e3306186c1898cd)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:38.737931 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:38.788177 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8158ebeedf7438b9e3306186c1898cd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:38.797274 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8158ebeedf7438b9e3306186c1898cd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:38.797887 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8158ebeedf7438b9e3306186c1898cd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:38.805735 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8158ebeedf7438b9e3306186c1898cd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:38.813792 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:38.819375 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8158ebeedf7438b9e3306186c1898cd'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:38.826481 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1456/2905] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:32:38 2022] DELETE /identity/v3/users/b8158ebeedf7438b9e3306186c1898cd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:32:38.999084 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:39.012752 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e29364-8039-41c1-a898-da952921cf5b service placement] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:39.045143 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=100 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:39.077729 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:39.078657 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:39.078935 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:39.079207 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] PATH_INFO: `/v3/projects/4bb5a1e0698348739f44376f82c933ad` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:39.080552 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb5a1e0698348739f44376f82c933ad)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:39.082500 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:39.131194 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb5a1e0698348739f44376f82c933ad', 'request_id': 'req-751aa544-d552-49cd-ac98-7b7e903de230'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.137328 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb5a1e0698348739f44376f82c933ad', 'request_id': 'req-751aa544-d552-49cd-ac98-7b7e903de230'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.142882 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb5a1e0698348739f44376f82c933ad', 'request_id': 'req-751aa544-d552-49cd-ac98-7b7e903de230'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.157256 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] The token cache is being invalidate because project 4bb5a1e0698348739f44376f82c933ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:32:39.157529 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.158430 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.160377 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1450/2906] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:32:38 2022] DELETE /identity/v3/projects/4bb5a1e0698348739f44376f82c933ad => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:32:39.164396 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:39.165869 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:39.179629 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6c2ac-580a-4000-a320-c1b8fd3a4349 admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:39.239091 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:39.241013 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:39.241335 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:39.241617 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] PATH_INFO: `/v3/users/58714b34e1bd4b76ae80b1a2293677da` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:39.242860 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=58714b34e1bd4b76ae80b1a2293677da)` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:39.244879 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:39.294167 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58714b34e1bd4b76ae80b1a2293677da'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.303307 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58714b34e1bd4b76ae80b1a2293677da'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.303956 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58714b34e1bd4b76ae80b1a2293677da'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.311180 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58714b34e1bd4b76ae80b1a2293677da'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.322784 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.notifications [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58714b34e1bd4b76ae80b1a2293677da'} {{(pid=80370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.328607 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1457/2907] 10.222.0.16 () {58 vars in 1223 bytes} [Mon Jul 4 04:32:39 2022] DELETE /identity/v3/users/58714b34e1bd4b76ae80b1a2293677da => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:32:39.520378 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:39.522131 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:39.538295 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa544-d552-49cd-ac98-7b7e903de230 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:39.597940 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '769a75081be74879911bd8c4f4cb57da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525d4bbd65cc40f382674c9a7977e83e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:39.599029 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:39.599422 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:39.599809 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] PATH_INFO: `/v3/projects/83178ea0925a4d0caae854926a2bec25` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:39.601827 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83178ea0925a4d0caae854926a2bec25)` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:39.604544 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:39.657075 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83178ea0925a4d0caae854926a2bec25', 'request_id': 'req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.663323 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83178ea0925a4d0caae854926a2bec25', 'request_id': 'req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.668967 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83178ea0925a4d0caae854926a2bec25', 'request_id': 'req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2'} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.670692 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:39.684201 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] The token cache is being invalidate because project 83178ea0925a4d0caae854926a2bec25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 04:32:39.684421 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.685060 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.notifications [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 04:32:39.686699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1451/2908] 10.222.0.16 () {58 vars in 1232 bytes} [Mon Jul 4 04:32:39 2022] DELETE /identity/v3/projects/83178ea0925a4d0caae854926a2bec25 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 04:32:49.325257 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:49.683625 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG dbcounter [-] [80371] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:32:53.974235 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Authenticating user token {{(pid=80370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:32:53.976542 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:54.055416 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6133f-9a95-4b1c-93d4-5de1237b7f1b admin admin] Validating token access rules against request {{(pid=80370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:32:54.115833 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:32:54.116478 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] REQUEST_METHOD: `GET` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:32:54.116763 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:32:54.117037 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:32:54.120688 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.fernet_utils [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:32:54.139374 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:32:54.140982 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962571a4-694b-4db3-b6a9-f5e6d49982d1 service placement] RBAC: Authorization granted {{(pid=80370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:32:54.151894 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: [pid: 80370|app: 0|req: 1458/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:32:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 04:33:04.146766 dv-cinder-742162-5-iscsi devstack@keystone.service[80370]: DEBUG dbcounter [-] [80370] Writing DB stats keystone:SELECT=58 {{(pid=80370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 04:37:55.676228 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Authenticating user token {{(pid=80371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 04:37:55.677704 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.fernet_utils [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 04:37:55.770159 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbda7b-4acf-4e65-ba0b-2cfdf4c237a2 admin admin] Validating token access rules against request {{(pid=80371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 04:37:55.775426 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01abfdc5d30744c89a476de93f537305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bea1bc67780a4c32882810f1c0ffcb6a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 04:37:55.776082 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] REQUEST_METHOD: `GET` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 04:37:55.776364 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] SCRIPT_NAME: `/identity` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 04:37:55.776640 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 04:37:55.785699 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 04:37:55.787311 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a816c-80d0-4320-a884-018f8d42e529 service placement] RBAC: Authorization granted {{(pid=80371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 04:37:55.797455 dv-cinder-742162-5-iscsi devstack@keystone.service[80371]: [pid: 80371|app: 0|req: 1452/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Mon Jul 4 04:37:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)